[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:22:46 dist-test-slave-2w3w krb5kdc[17179](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:22:46 dist-test-slave-2w3w krb5kdc[17179](info): set up 2 sockets
Aug 07 11:22:46 dist-test-slave-2w3w krb5kdc[17179](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 11:22:47.146643 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:22:47 dist-test-slave-2w3w krb5kdc[17179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:22:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:22:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:22:47.348693 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:38359
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:33529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:38359
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:22:47.973510 17192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:22:47.973845 17192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:22:47.973997 17192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:22:47.983901 17192 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:22:47.984053 17192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:22:47.984120 17192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:22:47.984257 17192 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:22:47.984341 17192 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:22:47.997896 17192 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:33529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:38359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:38359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:22:47.999409 17192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:22:48.000903 17192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:22:48.019217 17202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:22:48.019451 17199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:22:48.019953 17197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:22:48.021087 17192 server_base.cc:1034] running on GCE node
I20260807 11:22:48.022321 17192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:22:48.024477 17192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:22:48.025743 17192 hybrid_clock.cc:648] HybridClock initialized: now 1786101768025650 us; error 69 us; skew 500 ppm
I20260807 11:22:48.026165 17192 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:22:48 dist-test-slave-2w3w krb5kdc[17179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101768, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:22:48.031733 17192 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:22:48.034027 17192 webserver.cc:460] Webserver started at http://127.16.197.62:40125/ using document root <none> and password file <none>
I20260807 11:22:48.034891 17192 fs_manager.cc:359] Metadata directory not provided
I20260807 11:22:48.035127 17192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:22:48.035506 17192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:22:48.040352 17192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "27b776f788454a47893956d1efbdbfdc"
format_stamp: "Formatted at 2026-08-07 11:22:48 on dist-test-slave-2w3w"
server_key: "48bf1fb478c9afe9292248a3c6015994"
server_key_iv: "6b51aa76a540d72ceb96d8cd8b812ebf"
server_key_version: "encryptionkey@0"
I20260807 11:22:48.041246 17192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "27b776f788454a47893956d1efbdbfdc"
format_stamp: "Formatted at 2026-08-07 11:22:48 on dist-test-slave-2w3w"
server_key: "48bf1fb478c9afe9292248a3c6015994"
server_key_iv: "6b51aa76a540d72ceb96d8cd8b812ebf"
server_key_version: "encryptionkey@0"
I20260807 11:22:48.047955 17192 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 11:22:48.052704 17210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:22:48.054081 17192 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.001s
I20260807 11:22:48.054314 17192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "27b776f788454a47893956d1efbdbfdc"
format_stamp: "Formatted at 2026-08-07 11:22:48 on dist-test-slave-2w3w"
server_key: "48bf1fb478c9afe9292248a3c6015994"
server_key_iv: "6b51aa76a540d72ceb96d8cd8b812ebf"
server_key_version: "encryptionkey@0"
I20260807 11:22:48.054625 17192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:22:48.076108 17192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:22:48.081676 17192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:22:48.082083 17192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:22:48.122753 17192 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:38359
I20260807 11:22:48.122816 17262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:38359 every 8 connection(s)
I20260807 11:22:48.124660 17192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:22:48.126477 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 17192
I20260807 11:22:48.126950 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:22:48.129189 17172 external_mini_cluster.cc:1428] Setting key 6295359e52e385c303086289ec2b73be
I20260807 11:22:48.130271 17263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:22:48.146763 17263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27b776f788454a47893956d1efbdbfdc: Bootstrap starting.
I20260807 11:22:48.152297 17263 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27b776f788454a47893956d1efbdbfdc: Neither blocks nor log segments found. Creating new log.
I20260807 11:22:48.154817 17263 log.cc:824] T 00000000000000000000000000000000 P 27b776f788454a47893956d1efbdbfdc: Log is configured to *not* fsync() on all Append() calls
I20260807 11:22:48.159756 17263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27b776f788454a47893956d1efbdbfdc: No bootstrap required, opened a new log
I20260807 11:22:48.171196 17263 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27b776f788454a47893956d1efbdbfdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27b776f788454a47893956d1efbdbfdc" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 38359 } }
I20260807 11:22:48.171727 17263 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27b776f788454a47893956d1efbdbfdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:22:48.171911 17263 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27b776f788454a47893956d1efbdbfdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27b776f788454a47893956d1efbdbfdc, State: Initialized, Role: FOLLOWER
Aug 07 11:22:48 dist-test-slave-2w3w krb5kdc[17179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:22:48.172735 17263 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27b776f788454a47893956d1efbdbfdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27b776f788454a47893956d1efbdbfdc" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 38359 } }
I20260807 11:22:48.173110 17263 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27b776f788454a47893956d1efbdbfdc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:22:48.173324 17263 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27b776f788454a47893956d1efbdbfdc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:22:48.173573 17263 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27b776f788454a47893956d1efbdbfdc [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:22:48.176949 17263 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27b776f788454a47893956d1efbdbfdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27b776f788454a47893956d1efbdbfdc" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 38359 } }
I20260807 11:22:48.177898 17263 leader_election.cc:302] T 00000000000000000000000000000000 P 27b776f788454a47893956d1efbdbfdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27b776f788454a47893956d1efbdbfdc; no voters:
I20260807 11:22:48.179385 17263 leader_election.cc:288] T 00000000000000000000000000000000 P 27b776f788454a47893956d1efbdbfdc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:22:48.179560 17268 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27b776f788454a47893956d1efbdbfdc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:22:48.182149 17268 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27b776f788454a47893956d1efbdbfdc [term 1 LEADER]: Becoming Leader. State: Replica: 27b776f788454a47893956d1efbdbfdc, State: Running, Role: LEADER
I20260807 11:22:48.183130 17263 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27b776f788454a47893956d1efbdbfdc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:22:48.183076 17268 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27b776f788454a47893956d1efbdbfdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27b776f788454a47893956d1efbdbfdc" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 38359 } }
I20260807 11:22:48.186838 17269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27b776f788454a47893956d1efbdbfdc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27b776f788454a47893956d1efbdbfdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27b776f788454a47893956d1efbdbfdc" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 38359 } } }
I20260807 11:22:48.187736 17269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27b776f788454a47893956d1efbdbfdc [sys.catalog]: This master's current role is: LEADER
I20260807 11:22:48.188606 17270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27b776f788454a47893956d1efbdbfdc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27b776f788454a47893956d1efbdbfdc. Latest consensus state: current_term: 1 leader_uuid: "27b776f788454a47893956d1efbdbfdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27b776f788454a47893956d1efbdbfdc" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 38359 } } }
I20260807 11:22:48.189251 17270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27b776f788454a47893956d1efbdbfdc [sys.catalog]: This master's current role is: LEADER
I20260807 11:22:48.190865 17265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:48.134021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48762 (local address 127.16.197.62:38359)
0807 11:22:48.135644 (+ 1623us) server_negotiation.cc:206] Beginning negotiation
0807 11:22:48.135672 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0807 11:22:48.138711 (+ 3039us) server_negotiation.cc:407] Connection header received
0807 11:22:48.154517 (+ 15806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:22:48.154558 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:22:48.155210 (+ 652us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:22:48.155833 (+ 623us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:22:48.160629 (+ 4796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:48.163148 (+ 2519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:48.165454 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:48.166497 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:48.166769 (+ 272us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:48.174835 (+ 8066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:22:48.174848 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:22:48.174867 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:22:48.174936 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:22:48.180386 (+ 5450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:48.181878 (+ 1492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:48.181892 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:48.181903 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:48.182250 (+ 347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:48.182954 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:48.182984 (+ 30us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:48.182988 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:48.183881 (+ 893us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:22:48.184506 (+ 625us) server_negotiation.cc:1035] Waiting for connection context
0807 11:22:48.187600 (+ 3094us) server_negotiation.cc:299] Negotiation successful
0807 11:22:48.188232 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1252,"thread_start_us":1577,"threads_started":1}
I20260807 11:22:48.191113 17273 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:22:48.206118 17273 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:22:48.221544 17273 catalog_manager.cc:1347] Generated new cluster ID: ffbadce4ad374bc9b993afb969bde1e2
I20260807 11:22:48.221737 17273 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:22:48.245457 17273 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:22:48.247406 17273 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:22:48.262684 17273 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27b776f788454a47893956d1efbdbfdc: Generated new TSK 0
I20260807 11:22:48.263679 17273 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.197.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
I20260807 11:22:48.326833 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:0
--local_ip_for_outbound_sockets=127.16.197.1
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:38359
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:33529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:22:48.901504 17290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:22:48.901959 17290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:22:48.902226 17290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:22:48.912364 17290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:22:48.912588 17290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:22:48.912868 17290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:22:48.925740 17290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:33529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:38359
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:22:48.927433 17290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:22:48.929212 17290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:22:48.947854 17295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:22:48.948856 17296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:22:48.949117 17298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:22:48.950001 17290 server_base.cc:1034] running on GCE node
I20260807 11:22:48.951088 17290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:22:48.952319 17290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:22:48.953601 17290 hybrid_clock.cc:648] HybridClock initialized: now 1786101768953511 us; error 59 us; skew 500 ppm
I20260807 11:22:48.954048 17290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:22:48 dist-test-slave-2w3w krb5kdc[17179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101768, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:22:48.959070 17290 init.cc:377] Logged in from keytab as kudu/127.16.197.1@KRBTEST.COM (short username kudu)
I20260807 11:22:48.961539 17290 webserver.cc:460] Webserver started at http://127.16.197.1:38087/ using document root <none> and password file <none>
I20260807 11:22:48.962347 17290 fs_manager.cc:359] Metadata directory not provided
I20260807 11:22:48.962548 17290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:22:48.962953 17290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:22:48.966643 17290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-0/data/instance:
uuid: "3eb8476802734cb9bb61a844783e77b2"
format_stamp: "Formatted at 2026-08-07 11:22:48 on dist-test-slave-2w3w"
server_key: "3578ebfca774b7dc63201da475efb9ba"
server_key_iv: "40ee72908212145840fa5983c4aea73c"
server_key_version: "encryptionkey@0"
I20260807 11:22:48.967653 17290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance:
uuid: "3eb8476802734cb9bb61a844783e77b2"
format_stamp: "Formatted at 2026-08-07 11:22:48 on dist-test-slave-2w3w"
server_key: "3578ebfca774b7dc63201da475efb9ba"
server_key_iv: "40ee72908212145840fa5983c4aea73c"
server_key_version: "encryptionkey@0"
I20260807 11:22:48.974763 17290 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260807 11:22:48.980197 17305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:22:48.982499 17290 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260807 11:22:48.982722 17290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "3eb8476802734cb9bb61a844783e77b2"
format_stamp: "Formatted at 2026-08-07 11:22:48 on dist-test-slave-2w3w"
server_key: "3578ebfca774b7dc63201da475efb9ba"
server_key_iv: "40ee72908212145840fa5983c4aea73c"
server_key_version: "encryptionkey@0"
I20260807 11:22:48.983131 17290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:22:49.009948 17290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:22:49.016222 17290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:22:49.016634 17290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:22:49.018085 17290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:22:49.020015 17290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:22:49.020149 17290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:22:49.020397 17290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:22:49.020509 17290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:22:49.125488 17290 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:35785
I20260807 11:22:49.125984 17425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:35785 every 8 connection(s)
I20260807 11:22:49.127593 17290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:22:49.134524 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 17290
I20260807 11:22:49.135032 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance
I20260807 11:22:49.136168 17172 external_mini_cluster.cc:1428] Setting key 1f52c1d68d5e9df6490a378e5fc59390
Aug 07 11:22:49 dist-test-slave-2w3w krb5kdc[17179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101768, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:22:49.162652 17428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:49.132709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:37869 (local address 127.16.197.62:38359)
0807 11:22:49.133385 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0807 11:22:49.133392 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:22:49.135141 (+ 1749us) server_negotiation.cc:407] Connection header received
0807 11:22:49.138192 (+ 3051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:22:49.138198 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:22:49.138275 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:22:49.138358 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:22:49.141072 (+ 2714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:49.142625 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:49.145579 (+ 2954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:49.146325 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:49.146410 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:49.152132 (+ 5722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:22:49.152139 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:22:49.152144 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:22:49.152205 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:22:49.158132 (+ 5927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:49.159552 (+ 1420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:49.159558 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:49.159563 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:49.159769 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:49.160402 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:49.160406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:49.160408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:49.160812 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:22:49.160992 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 11:22:49.161868 (+ 876us) server_negotiation.cc:299] Negotiation successful
0807 11:22:49.162121 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":384,"threads_started":1}
I20260807 11:22:49.165153 17429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:49.133477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:38359 (local address 127.16.197.1:37869)
0807 11:22:49.134818 (+ 1341us) negotiation.cc:107] Waiting for socket to connect
0807 11:22:49.134930 (+ 112us) client_negotiation.cc:174] Beginning negotiation
0807 11:22:49.137836 (+ 2906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:22:49.139016 (+ 1180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:22:49.139030 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:22:49.139867 (+ 837us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:22:49.140785 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:22:49.140819 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:22:49.142846 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:22:49.142852 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:22:49.145392 (+ 2540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:22:49.145409 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:22:49.146871 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:22:49.146876 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:22:49.147298 (+ 422us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:49.148410 (+ 1112us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:22:49.148445 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:22:49.151757 (+ 3312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:22:49.158363 (+ 6606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:22:49.158377 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:22:49.158419 (+ 42us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:22:49.159371 (+ 952us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:22:49.159947 (+ 576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:22:49.159950 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:22:49.159952 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:22:49.160194 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:22:49.160965 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:22:49.160974 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:22:49.161362 (+ 388us) client_negotiation.cc:769] Sending connection context
0807 11:22:49.163001 (+ 1639us) client_negotiation.cc:240] Negotiation successful
0807 11:22:49.163676 (+ 675us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":942,"thread_start_us":587,"threads_started":1}
I20260807 11:22:49.167109 17426 heartbeater.cc:347] Connected to a master server at 127.16.197.62:38359
I20260807 11:22:49.167487 17426 heartbeater.cc:464] Registering TS with master...
I20260807 11:22:49.168361 17426 heartbeater.cc:511] Master 127.16.197.62:38359 requested a full tablet report, sending...
I20260807 11:22:49.171393 17227 ts_manager.cc:194] Registered new tserver with Master: 3eb8476802734cb9bb61a844783e77b2 (127.16.197.1:35785)
WARNING: no policy specified for kudu/127.16.197.2@KRBTEST.COM; defaulting to no policy
I20260807 11:22:49.173451 17227 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.1@KRBTEST.COM'} at 127.16.197.1:37869
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
I20260807 11:22:49.206840 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:0
--local_ip_for_outbound_sockets=127.16.197.2
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:38359
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:33529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:22:49.882133 17432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:22:49.882656 17432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:22:49.882923 17432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:22:49.893359 17432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:22:49.893637 17432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:22:49.893906 17432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:22:49.907196 17432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:33529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:38359
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:22:49.908887 17432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:22:49.910744 17432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:22:49.928877 17437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:22:49.929576 17438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:22:49.931192 17432 server_base.cc:1034] running on GCE node
W20260807 11:22:49.931988 17440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:22:49.932708 17432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:22:49.933933 17432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:22:49.935314 17432 hybrid_clock.cc:648] HybridClock initialized: now 1786101769935162 us; error 60 us; skew 500 ppm
I20260807 11:22:49.935932 17432 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:22:49 dist-test-slave-2w3w krb5kdc[17179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101769, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:22:49.942030 17432 init.cc:377] Logged in from keytab as kudu/127.16.197.2@KRBTEST.COM (short username kudu)
I20260807 11:22:49.944783 17432 webserver.cc:460] Webserver started at http://127.16.197.2:45941/ using document root <none> and password file <none>
I20260807 11:22:49.945897 17432 fs_manager.cc:359] Metadata directory not provided
I20260807 11:22:49.946159 17432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:22:49.946617 17432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:22:49.950712 17432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-1/data/instance:
uuid: "5166711ad2ab4996b78172cd01649012"
format_stamp: "Formatted at 2026-08-07 11:22:49 on dist-test-slave-2w3w"
server_key: "926a3c3e7483d9d3505a38b7c7c82e09"
server_key_iv: "9e526a94e2e85c480e687c7885ab6449"
server_key_version: "encryptionkey@0"
I20260807 11:22:49.952099 17432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance:
uuid: "5166711ad2ab4996b78172cd01649012"
format_stamp: "Formatted at 2026-08-07 11:22:49 on dist-test-slave-2w3w"
server_key: "926a3c3e7483d9d3505a38b7c7c82e09"
server_key_iv: "9e526a94e2e85c480e687c7885ab6449"
server_key_version: "encryptionkey@0"
I20260807 11:22:49.959836 17432 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.006s
I20260807 11:22:49.964653 17447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:22:49.966255 17432 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260807 11:22:49.966475 17432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "5166711ad2ab4996b78172cd01649012"
format_stamp: "Formatted at 2026-08-07 11:22:49 on dist-test-slave-2w3w"
server_key: "926a3c3e7483d9d3505a38b7c7c82e09"
server_key_iv: "9e526a94e2e85c480e687c7885ab6449"
server_key_version: "encryptionkey@0"
I20260807 11:22:49.966790 17432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:22:49.989920 17432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:22:49.995357 17432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:22:49.995834 17432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:22:49.997563 17432 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:22:50.000058 17432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:22:50.000241 17432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:22:50.000502 17432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:22:50.000659 17432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:22:50.090276 17432 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:44843
I20260807 11:22:50.090467 17570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:44843 every 8 connection(s)
I20260807 11:22:50.092183 17432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:22:50.101089 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 17432
I20260807 11:22:50.101431 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance
I20260807 11:22:50.102219 17172 external_mini_cluster.cc:1428] Setting key b84016145ea9f3f97a70129dede20423
Aug 07 11:22:50 dist-test-slave-2w3w krb5kdc[17179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101769, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:22:50.129031 17575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:50.096889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:38359 (local address 127.16.197.2:55367)
0807 11:22:50.098159 (+ 1270us) negotiation.cc:107] Waiting for socket to connect
0807 11:22:50.098200 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 11:22:50.101387 (+ 3187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:22:50.102520 (+ 1133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:22:50.102531 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:22:50.103552 (+ 1021us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:22:50.104677 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:22:50.104701 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:22:50.106943 (+ 2242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:22:50.106947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:22:50.108836 (+ 1889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:22:50.108850 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:22:50.110779 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:22:50.110785 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:22:50.111194 (+ 409us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:50.112440 (+ 1246us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:22:50.112465 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:22:50.115918 (+ 3453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:22:50.122555 (+ 6637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:22:50.122563 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:22:50.122579 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:22:50.123321 (+ 742us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:22:50.124009 (+ 688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:22:50.124014 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:22:50.124018 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:22:50.124444 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:22:50.125458 (+ 1014us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:22:50.125466 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:22:50.126095 (+ 629us) client_negotiation.cc:769] Sending connection context
0807 11:22:50.126423 (+ 328us) client_negotiation.cc:240] Negotiation successful
0807 11:22:50.127176 (+ 753us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":988,"thread_start_us":641,"threads_started":1}
I20260807 11:22:50.130831 17574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:50.095882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:55367 (local address 127.16.197.62:38359)
0807 11:22:50.096682 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0807 11:22:50.096688 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:22:50.098485 (+ 1797us) server_negotiation.cc:407] Connection header received
0807 11:22:50.101808 (+ 3323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:22:50.101815 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:22:50.101930 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:22:50.102161 (+ 231us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:22:50.104990 (+ 2829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:50.106678 (+ 1688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:50.109070 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:50.110433 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:50.110631 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:50.116384 (+ 5753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:22:50.116404 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:22:50.116411 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:22:50.116465 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:22:50.122338 (+ 5873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:50.123504 (+ 1166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:50.123511 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:50.123516 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:50.123784 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:50.124611 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:50.124616 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:50.124619 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:50.125239 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:22:50.129693 (+ 4454us) server_negotiation.cc:1035] Waiting for connection context
0807 11:22:50.129970 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 11:22:50.130235 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":500,"threads_started":1}
I20260807 11:22:50.132341 17572 heartbeater.cc:347] Connected to a master server at 127.16.197.62:38359
I20260807 11:22:50.132730 17572 heartbeater.cc:464] Registering TS with master...
I20260807 11:22:50.133682 17572 heartbeater.cc:511] Master 127.16.197.62:38359 requested a full tablet report, sending...
I20260807 11:22:50.135623 17227 ts_manager.cc:194] Registered new tserver with Master: 5166711ad2ab4996b78172cd01649012 (127.16.197.2:44843)
I20260807 11:22:50.136672 17227 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.2@KRBTEST.COM'} at 127.16.197.2:55367
WARNING: no policy specified for kudu/127.16.197.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
I20260807 11:22:50.174616 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:0
--local_ip_for_outbound_sockets=127.16.197.3
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:38359
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:33529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 11:22:50.178265 17426 heartbeater.cc:503] Master 127.16.197.62:38359 was elected leader, sending a full tablet report...
W20260807 11:22:50.762094 17578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:22:50.762432 17578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:22:50.762594 17578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:22:50.772617 17578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:22:50.772805 17578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:22:50.773028 17578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:22:50.787535 17578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:33529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:38359
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:22:50.789117 17578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:22:50.790863 17578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:22:50.811296 17585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:22:50.811311 17587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:22:50.811877 17584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:22:50.813112 17578 server_base.cc:1034] running on GCE node
I20260807 11:22:50.814147 17578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:22:50.815392 17578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:22:50.816658 17578 hybrid_clock.cc:648] HybridClock initialized: now 1786101770816586 us; error 44 us; skew 500 ppm
I20260807 11:22:50.817039 17578 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:22:50 dist-test-slave-2w3w krb5kdc[17179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101770, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:22:50.820966 17578 init.cc:377] Logged in from keytab as kudu/127.16.197.3@KRBTEST.COM (short username kudu)
I20260807 11:22:50.822821 17578 webserver.cc:460] Webserver started at http://127.16.197.3:37319/ using document root <none> and password file <none>
I20260807 11:22:50.823647 17578 fs_manager.cc:359] Metadata directory not provided
I20260807 11:22:50.823822 17578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:22:50.824151 17578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:22:50.827487 17578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-2/data/instance:
uuid: "e84b6604b9b048c99ed50c5f4890e6fe"
format_stamp: "Formatted at 2026-08-07 11:22:50 on dist-test-slave-2w3w"
server_key: "165d6394d64cf2514a8686b99268859a"
server_key_iv: "e9565bdbda5a6eed20792cddf6b74be4"
server_key_version: "encryptionkey@0"
I20260807 11:22:50.828394 17578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance:
uuid: "e84b6604b9b048c99ed50c5f4890e6fe"
format_stamp: "Formatted at 2026-08-07 11:22:50 on dist-test-slave-2w3w"
server_key: "165d6394d64cf2514a8686b99268859a"
server_key_iv: "e9565bdbda5a6eed20792cddf6b74be4"
server_key_version: "encryptionkey@0"
I20260807 11:22:50.834084 17578 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 11:22:50.838471 17594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:22:50.839975 17578 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 11:22:50.840169 17578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "e84b6604b9b048c99ed50c5f4890e6fe"
format_stamp: "Formatted at 2026-08-07 11:22:50 on dist-test-slave-2w3w"
server_key: "165d6394d64cf2514a8686b99268859a"
server_key_iv: "e9565bdbda5a6eed20792cddf6b74be4"
server_key_version: "encryptionkey@0"
I20260807 11:22:50.840418 17578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:22:50.855346 17578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:22:50.860474 17578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:22:50.860870 17578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:22:50.862102 17578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:22:50.863901 17578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:22:50.864032 17578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:22:50.864221 17578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:22:50.864322 17578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:22:50.931778 17578 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:44489
I20260807 11:22:50.931890 17712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:44489 every 8 connection(s)
I20260807 11:22:50.933219 17578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:22:50.935309 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 17578
I20260807 11:22:50.935685 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance
I20260807 11:22:50.936470 17172 external_mini_cluster.cc:1428] Setting key 3c7749befc66d87b60acac93b842afb0
Aug 07 11:22:50 dist-test-slave-2w3w krb5kdc[17179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101770, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:22:50.958942 17716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:50.937176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:55271 (local address 127.16.197.62:38359)
0807 11:22:50.938565 (+ 1389us) server_negotiation.cc:206] Beginning negotiation
0807 11:22:50.938571 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:22:50.940066 (+ 1495us) server_negotiation.cc:407] Connection header received
0807 11:22:50.942526 (+ 2460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:22:50.942530 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:22:50.942642 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:22:50.942880 (+ 238us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:22:50.944704 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:50.945605 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:50.947409 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:50.947888 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:50.948026 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:50.952265 (+ 4239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:22:50.952271 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:22:50.952274 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:22:50.952309 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:22:50.955880 (+ 3571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:50.956615 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:50.956618 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:50.956620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:50.956750 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:50.957161 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:50.957164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:50.957166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:50.957474 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:22:50.957604 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 11:22:50.958384 (+ 780us) server_negotiation.cc:299] Negotiation successful
0807 11:22:50.958575 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1254,"thread_start_us":1003,"threads_started":1}
I20260807 11:22:50.960148 17717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:50.938000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:38359 (local address 127.16.197.3:55271)
0807 11:22:50.939707 (+ 1707us) negotiation.cc:107] Waiting for socket to connect
0807 11:22:50.939767 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 11:22:50.942257 (+ 2490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:22:50.943227 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:22:50.943236 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:22:50.943788 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:22:50.944529 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:22:50.944549 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:22:50.945772 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:22:50.945776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:22:50.947267 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:22:50.947280 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:22:50.948161 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:22:50.948169 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:22:50.948425 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:50.949133 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:22:50.949166 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:22:50.951952 (+ 2786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:22:50.956013 (+ 4061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:22:50.956021 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:22:50.956040 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:22:50.956504 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:22:50.956848 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:22:50.956851 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:22:50.956853 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:22:50.957059 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:22:50.957602 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:22:50.957608 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:22:50.957980 (+ 372us) client_negotiation.cc:769] Sending connection context
0807 11:22:50.958226 (+ 246us) client_negotiation.cc:240] Negotiation successful
0807 11:22:50.958678 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1322,"thread_start_us":852,"threads_started":1}
I20260807 11:22:50.961752 17713 heartbeater.cc:347] Connected to a master server at 127.16.197.62:38359
I20260807 11:22:50.962148 17713 heartbeater.cc:464] Registering TS with master...
I20260807 11:22:50.963224 17713 heartbeater.cc:511] Master 127.16.197.62:38359 requested a full tablet report, sending...
I20260807 11:22:50.965087 17227 ts_manager.cc:194] Registered new tserver with Master: e84b6604b9b048c99ed50c5f4890e6fe (127.16.197.3:44489)
I20260807 11:22:50.966456 17227 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.3@KRBTEST.COM'} at 127.16.197.3:55271
I20260807 11:22:50.966871 17172 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 11:22:50 dist-test-slave-2w3w krb5kdc[17179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101770, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 11:22:51 dist-test-slave-2w3w krb5kdc[17179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101770, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.197.1@KRBTEST.COM
I20260807 11:22:51.028041 17722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:51.008904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46408 (local address 127.16.197.1:35785)
0807 11:22:51.009714 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0807 11:22:51.009720 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:22:51.009864 (+ 144us) server_negotiation.cc:407] Connection header received
0807 11:22:51.010021 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:22:51.010029 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:22:51.010254 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:22:51.010529 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:22:51.012729 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:51.013754 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:51.015478 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:51.016086 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:51.016200 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:51.021347 (+ 5147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:22:51.021361 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:22:51.021375 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:22:51.021415 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:22:51.024608 (+ 3193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:51.025444 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:51.025452 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:51.025454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:51.025689 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:51.026220 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:51.026224 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:51.026226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:51.026668 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:22:51.027131 (+ 463us) server_negotiation.cc:1035] Waiting for connection context
0807 11:22:51.027298 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 11:22:51.027520 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":427,"threads_started":1}
W20260807 11:22:51.029646 17334 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46408
Aug 07 11:22:51 dist-test-slave-2w3w krb5kdc[17179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 11:22:51 dist-test-slave-2w3w krb5kdc[17179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.1@KRBTEST.COM
I20260807 11:22:51.139958 17572 heartbeater.cc:503] Master 127.16.197.62:38359 was elected leader, sending a full tablet report...
I20260807 11:22:51.970010 17713 heartbeater.cc:503] Master 127.16.197.62:38359 was elected leader, sending a full tablet report...
W20260807 11:22:52.001829 17722 sasl_common.cc:323] Time spent calling sasl_server_start(): real 0.919s user 0.000s sys 0.002s
I20260807 11:22:52.004933 17722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:51.073523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46410 (local address 127.16.197.1:35785)
0807 11:22:51.073854 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0807 11:22:51.073862 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:22:51.074134 (+ 272us) server_negotiation.cc:407] Connection header received
0807 11:22:51.074370 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:22:51.074377 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:22:51.074458 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:22:51.074574 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:22:51.076056 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:51.076779 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:51.078016 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:51.078449 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:51.078590 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:51.082527 (+ 3937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:22:51.082532 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:22:51.082536 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:22:51.082570 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:22:52.002020 (+919450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:52.003038 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:52.003044 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:52.003046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:52.003208 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:52.003657 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:52.003661 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:52.003663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:52.003952 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:22:52.004197 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 11:22:52.004477 (+ 280us) server_negotiation.cc:299] Negotiation successful
0807 11:22:52.004657 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260807 11:22:52.006846 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 17290
I20260807 11:22:52.021019 17722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:52.007940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46420 (local address 127.16.197.1:35785)
0807 11:22:52.008099 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 11:22:52.008104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:22:52.008439 (+ 335us) server_negotiation.cc:407] Connection header received
0807 11:22:52.008563 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:22:52.008567 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:22:52.008621 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:22:52.008702 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:22:52.009860 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:52.010694 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:52.012070 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:52.012705 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:52.012839 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:52.015238 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:22:52.015244 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:22:52.015247 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:22:52.015282 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:22:52.018513 (+ 3231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:52.019268 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:52.019274 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:52.019279 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:52.019453 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:52.019793 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:52.019797 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:52.019799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:52.020076 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:22:52.020269 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 11:22:52.020495 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 11:22:52.020687 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 11:22:52.021910 17420 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:22:52.107656 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 17290
I20260807 11:22:52.126852 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 17432
Aug 07 11:22:52 dist-test-slave-2w3w krb5kdc[17179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.2@KRBTEST.COM
I20260807 11:22:52.143599 17730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:52.127781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37288 (local address 127.16.197.2:44843)
0807 11:22:52.128418 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0807 11:22:52.128425 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:22:52.128453 (+ 28us) server_negotiation.cc:407] Connection header received
0807 11:22:52.128513 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:22:52.128520 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:22:52.128724 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:22:52.128857 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:22:52.130092 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:52.131205 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:52.132480 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:52.133036 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:52.133134 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:52.137541 (+ 4407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:22:52.137555 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:22:52.137570 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:22:52.137607 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:22:52.140731 (+ 3124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:52.141409 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:52.141416 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:52.141418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:52.141540 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:52.141933 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:52.141938 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:52.141941 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:52.142363 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:22:52.142725 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0807 11:22:52.143020 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 11:22:52.143243 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":301,"threads_started":1}
I20260807 11:22:52.145068 17565 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:22:52.231434 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 17432
I20260807 11:22:52.250072 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 17578
Aug 07 11:22:52 dist-test-slave-2w3w krb5kdc[17179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.3@KRBTEST.COM
I20260807 11:22:52.266553 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:52.251162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38304 (local address 127.16.197.3:44489)
0807 11:22:52.251755 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0807 11:22:52.251764 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:22:52.251790 (+ 26us) server_negotiation.cc:407] Connection header received
0807 11:22:52.251849 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:22:52.251856 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:22:52.252078 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:22:52.252201 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:22:52.253470 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:52.254483 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:52.255783 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:52.256355 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:52.256455 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:52.260830 (+ 4375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:22:52.260842 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:22:52.260857 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:22:52.260891 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:22:52.263909 (+ 3018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:52.264526 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:52.264533 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:52.264535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:52.264656 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:52.264980 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:52.264983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:52.264985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:52.265337 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:22:52.265574 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 11:22:52.265851 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 11:22:52.266066 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":297,"threads_started":1}
I20260807 11:22:52.268132 17708 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:22:52.353192 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 17578
I20260807 11:22:52.371291 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 17192
I20260807 11:22:52.372500 17258 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:22:52.478454 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 17192
2026-08-07T11:22:52Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (6505 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:22:52 dist-test-slave-2w3w krb5kdc[17737](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:22:52 dist-test-slave-2w3w krb5kdc[17737](info): set up 2 sockets
Aug 07 11:22:52 dist-test-slave-2w3w krb5kdc[17737](info): commencing operation
krb5kdc: starting...
W20260807 11:22:53.608923 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:22:53 dist-test-slave-2w3w krb5kdc[17737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:22:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:22:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:22:53.806524 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:35715
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:39285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:35715
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:22:54.133642 17750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:22:54.134014 17750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:22:54.134178 17750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:22:54.141389 17750 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:22:54.141542 17750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:22:54.141655 17750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:22:54.141768 17750 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:22:54.141870 17750 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:22:54.151685 17750 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:39285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:35715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:35715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:22:54.153148 17750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:22:54.154510 17750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:22:54.166697 17758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:22:54.166669 17755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:22:54.166880 17756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:22:54.167500 17750 server_base.cc:1034] running on GCE node
I20260807 11:22:54.168231 17750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:22:54.169658 17750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:22:54.170900 17750 hybrid_clock.cc:648] HybridClock initialized: now 1786101774170827 us; error 44 us; skew 500 ppm
I20260807 11:22:54.171293 17750 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:22:54 dist-test-slave-2w3w krb5kdc[17737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101774, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:22:54.174451 17750 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:22:54.176092 17750 webserver.cc:460] Webserver started at http://127.16.197.62:43053/ using document root <none> and password file <none>
I20260807 11:22:54.176805 17750 fs_manager.cc:359] Metadata directory not provided
I20260807 11:22:54.176962 17750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:22:54.177284 17750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:22:54.180379 17750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "928ed62f472d44e2a476c740c772322c"
format_stamp: "Formatted at 2026-08-07 11:22:54 on dist-test-slave-2w3w"
server_key: "141f479930df3eb9b72fd0780fb91670"
server_key_iv: "b0999df78f997239c11726fe8975bcea"
server_key_version: "encryptionkey@0"
I20260807 11:22:54.181165 17750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "928ed62f472d44e2a476c740c772322c"
format_stamp: "Formatted at 2026-08-07 11:22:54 on dist-test-slave-2w3w"
server_key: "141f479930df3eb9b72fd0780fb91670"
server_key_iv: "b0999df78f997239c11726fe8975bcea"
server_key_version: "encryptionkey@0"
I20260807 11:22:54.186334 17750 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 11:22:54.189891 17765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:22:54.191424 17750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:22:54.191618 17750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "928ed62f472d44e2a476c740c772322c"
format_stamp: "Formatted at 2026-08-07 11:22:54 on dist-test-slave-2w3w"
server_key: "141f479930df3eb9b72fd0780fb91670"
server_key_iv: "b0999df78f997239c11726fe8975bcea"
server_key_version: "encryptionkey@0"
I20260807 11:22:54.191824 17750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:22:54.206738 17750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:22:54.210950 17750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:22:54.211310 17750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:22:54.231802 17750 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:35715
I20260807 11:22:54.231833 17821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:35715 every 8 connection(s)
I20260807 11:22:54.233246 17750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:22:54.235273 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 17750
I20260807 11:22:54.235515 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:22:54.236162 17172 external_mini_cluster.cc:1428] Setting key 3e356db31af514939d05fa5225933c5a
I20260807 11:22:54.237479 17822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:22:54.251722 17822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 928ed62f472d44e2a476c740c772322c: Bootstrap starting.
Aug 07 11:22:54 dist-test-slave-2w3w krb5kdc[17737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:22:54.255250 17822 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 928ed62f472d44e2a476c740c772322c: Neither blocks nor log segments found. Creating new log.
I20260807 11:22:54.256307 17822 log.cc:824] T 00000000000000000000000000000000 P 928ed62f472d44e2a476c740c772322c: Log is configured to *not* fsync() on all Append() calls
I20260807 11:22:54.259478 17822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 928ed62f472d44e2a476c740c772322c: No bootstrap required, opened a new log
I20260807 11:22:54.263844 17825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:54.238098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57102 (local address 127.16.197.62:35715)
0807 11:22:54.238901 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0807 11:22:54.238913 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 11:22:54.243588 (+ 4675us) server_negotiation.cc:407] Connection header received
0807 11:22:54.245464 (+ 1876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:22:54.245488 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:22:54.245883 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:22:54.246283 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:22:54.247908 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:54.249217 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:54.250512 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:54.251280 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:54.251485 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:54.255888 (+ 4403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:22:54.255898 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:22:54.255914 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:22:54.255950 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:22:54.259498 (+ 3548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:54.260223 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:54.260230 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:54.260236 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:54.260408 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:54.260838 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:54.260841 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:54.260843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:54.261425 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:22:54.261769 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0807 11:22:54.262301 (+ 532us) server_negotiation.cc:299] Negotiation successful
0807 11:22:54.262727 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":382,"threads_started":1}
I20260807 11:22:54.267755 17822 raft_consensus.cc:348] T 00000000000000000000000000000000 P 928ed62f472d44e2a476c740c772322c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928ed62f472d44e2a476c740c772322c" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 35715 } }
I20260807 11:22:54.268177 17822 raft_consensus.cc:374] T 00000000000000000000000000000000 P 928ed62f472d44e2a476c740c772322c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:22:54.268302 17822 raft_consensus.cc:729] T 00000000000000000000000000000000 P 928ed62f472d44e2a476c740c772322c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 928ed62f472d44e2a476c740c772322c, State: Initialized, Role: FOLLOWER
I20260807 11:22:54.268836 17822 consensus_queue.cc:260] T 00000000000000000000000000000000 P 928ed62f472d44e2a476c740c772322c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928ed62f472d44e2a476c740c772322c" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 35715 } }
I20260807 11:22:54.269111 17822 raft_consensus.cc:388] T 00000000000000000000000000000000 P 928ed62f472d44e2a476c740c772322c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:22:54.269258 17822 raft_consensus.cc:482] T 00000000000000000000000000000000 P 928ed62f472d44e2a476c740c772322c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:22:54.269476 17822 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 928ed62f472d44e2a476c740c772322c [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:22:54.271972 17822 raft_consensus.cc:504] T 00000000000000000000000000000000 P 928ed62f472d44e2a476c740c772322c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928ed62f472d44e2a476c740c772322c" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 35715 } }
I20260807 11:22:54.272493 17822 leader_election.cc:302] T 00000000000000000000000000000000 P 928ed62f472d44e2a476c740c772322c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 928ed62f472d44e2a476c740c772322c; no voters:
I20260807 11:22:54.273097 17822 leader_election.cc:288] T 00000000000000000000000000000000 P 928ed62f472d44e2a476c740c772322c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:22:54.273257 17827 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 928ed62f472d44e2a476c740c772322c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:22:54.273999 17827 raft_consensus.cc:686] T 00000000000000000000000000000000 P 928ed62f472d44e2a476c740c772322c [term 1 LEADER]: Becoming Leader. State: Replica: 928ed62f472d44e2a476c740c772322c, State: Running, Role: LEADER
I20260807 11:22:54.274606 17827 consensus_queue.cc:237] T 00000000000000000000000000000000 P 928ed62f472d44e2a476c740c772322c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928ed62f472d44e2a476c740c772322c" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 35715 } }
I20260807 11:22:54.275451 17822 sys_catalog.cc:564] T 00000000000000000000000000000000 P 928ed62f472d44e2a476c740c772322c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:22:54.277597 17828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 928ed62f472d44e2a476c740c772322c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "928ed62f472d44e2a476c740c772322c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928ed62f472d44e2a476c740c772322c" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 35715 } } }
I20260807 11:22:54.277558 17829 sys_catalog.cc:455] T 00000000000000000000000000000000 P 928ed62f472d44e2a476c740c772322c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 928ed62f472d44e2a476c740c772322c. Latest consensus state: current_term: 1 leader_uuid: "928ed62f472d44e2a476c740c772322c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928ed62f472d44e2a476c740c772322c" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 35715 } } }
I20260807 11:22:54.277998 17828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 928ed62f472d44e2a476c740c772322c [sys.catalog]: This master's current role is: LEADER
I20260807 11:22:54.278105 17829 sys_catalog.cc:458] T 00000000000000000000000000000000 P 928ed62f472d44e2a476c740c772322c [sys.catalog]: This master's current role is: LEADER
I20260807 11:22:54.278784 17832 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:22:54.284027 17832 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:22:54.294437 17832 catalog_manager.cc:1347] Generated new cluster ID: 14d03b53f0c94674b728cc51a3c16544
I20260807 11:22:54.294562 17832 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:22:54.321295 17832 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:22:54.322798 17832 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:22:54.333073 17832 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 928ed62f472d44e2a476c740c772322c: Generated new TSK 0
I20260807 11:22:54.333973 17832 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.197.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
I20260807 11:22:54.388780 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:0
--local_ip_for_outbound_sockets=127.16.197.1
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:35715
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:39285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:22:54.723158 17848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:22:54.723497 17848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:22:54.723680 17848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:22:54.730857 17848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:22:54.731040 17848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:22:54.731235 17848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:22:54.741317 17848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:39285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:35715
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:22:54.742694 17848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:22:54.744118 17848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:22:54.757884 17853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:22:54.757917 17854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:22:54.757965 17857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:22:54.758780 17848 server_base.cc:1034] running on GCE node
I20260807 11:22:54.759534 17848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:22:54.760543 17848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:22:54.761744 17848 hybrid_clock.cc:648] HybridClock initialized: now 1786101774761700 us; error 39 us; skew 500 ppm
I20260807 11:22:54.762079 17848 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:22:54 dist-test-slave-2w3w krb5kdc[17737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101774, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:22:54.765391 17848 init.cc:377] Logged in from keytab as kudu/127.16.197.1@KRBTEST.COM (short username kudu)
I20260807 11:22:54.766878 17848 webserver.cc:460] Webserver started at http://127.16.197.1:37341/ using document root <none> and password file <none>
I20260807 11:22:54.767623 17848 fs_manager.cc:359] Metadata directory not provided
I20260807 11:22:54.767725 17848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:22:54.767982 17848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:22:54.771028 17848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-0/data/instance:
uuid: "6ff057bf28ac40c3bdd645ffe7dc1a21"
format_stamp: "Formatted at 2026-08-07 11:22:54 on dist-test-slave-2w3w"
server_key: "4f9c314f9d4267148442b2b4c1e99e15"
server_key_iv: "63f788330fceeb9d14f967bb63cf22d7"
server_key_version: "encryptionkey@0"
I20260807 11:22:54.771804 17848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance:
uuid: "6ff057bf28ac40c3bdd645ffe7dc1a21"
format_stamp: "Formatted at 2026-08-07 11:22:54 on dist-test-slave-2w3w"
server_key: "4f9c314f9d4267148442b2b4c1e99e15"
server_key_iv: "63f788330fceeb9d14f967bb63cf22d7"
server_key_version: "encryptionkey@0"
I20260807 11:22:54.776947 17848 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 11:22:54.780437 17864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:22:54.781719 17848 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 11:22:54.781881 17848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "6ff057bf28ac40c3bdd645ffe7dc1a21"
format_stamp: "Formatted at 2026-08-07 11:22:54 on dist-test-slave-2w3w"
server_key: "4f9c314f9d4267148442b2b4c1e99e15"
server_key_iv: "63f788330fceeb9d14f967bb63cf22d7"
server_key_version: "encryptionkey@0"
I20260807 11:22:54.782069 17848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:22:54.802799 17848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:22:54.807502 17848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:22:54.807799 17848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:22:54.808812 17848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:22:54.810164 17848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:22:54.810281 17848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:22:54.810439 17848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:22:54.810510 17848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:22:54.846549 17848 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:44067
I20260807 11:22:54.846647 17981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:44067 every 8 connection(s)
I20260807 11:22:54.847908 17848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:22:54.851603 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 17848
I20260807 11:22:54.851859 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance
I20260807 11:22:54.852578 17172 external_mini_cluster.cc:1428] Setting key 65b61b65b7684d3eae68989eebc3b43f
Aug 07 11:22:54 dist-test-slave-2w3w krb5kdc[17737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101774, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:22:54.869962 17984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:54.850496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:40801 (local address 127.16.197.62:35715)
0807 11:22:54.851048 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0807 11:22:54.851054 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:22:54.851978 (+ 924us) server_negotiation.cc:407] Connection header received
0807 11:22:54.854074 (+ 2096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:22:54.854080 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:22:54.854150 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:22:54.854288 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:22:54.855960 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:54.856699 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:54.858252 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:54.858654 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:54.858816 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:54.862390 (+ 3574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:22:54.862399 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:22:54.862406 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:22:54.862454 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:22:54.866250 (+ 3796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:54.867195 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:54.867203 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:54.867210 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:54.867390 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:54.867990 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:54.867995 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:54.867998 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:54.868357 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:22:54.868528 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 11:22:54.869333 (+ 805us) server_negotiation.cc:299] Negotiation successful
0807 11:22:54.869577 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":273,"threads_started":1}
I20260807 11:22:54.871126 17985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:54.850889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:35715 (local address 127.16.197.1:40801)
0807 11:22:54.851755 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0807 11:22:54.851819 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 11:22:54.853813 (+ 1994us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:22:54.854553 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:22:54.854562 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:22:54.855211 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:22:54.855786 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:22:54.855803 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:22:54.856852 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:22:54.856862 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:22:54.858094 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:22:54.858107 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:22:54.858919 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:22:54.858928 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:22:54.859207 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:54.859935 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:22:54.859962 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:22:54.862161 (+ 2199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:22:54.866436 (+ 4275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:22:54.866443 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:22:54.866458 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:22:54.867034 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:22:54.867519 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:22:54.867522 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:22:54.867525 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:22:54.867816 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:22:54.868504 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:22:54.868513 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:22:54.868913 (+ 400us) client_negotiation.cc:769] Sending connection context
0807 11:22:54.869202 (+ 289us) client_negotiation.cc:240] Negotiation successful
0807 11:22:54.869827 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":334,"threads_started":1}
WARNING: no policy specified for kudu/127.16.197.2@KRBTEST.COM; defaulting to no policy
I20260807 11:22:54.873072 17982 heartbeater.cc:347] Connected to a master server at 127.16.197.62:35715
I20260807 11:22:54.873369 17982 heartbeater.cc:464] Registering TS with master...
I20260807 11:22:54.874148 17982 heartbeater.cc:511] Master 127.16.197.62:35715 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.2@KRBTEST.COM" created.
I20260807 11:22:54.876854 17782 ts_manager.cc:194] Registered new tserver with Master: 6ff057bf28ac40c3bdd645ffe7dc1a21 (127.16.197.1:44067)
I20260807 11:22:54.878360 17782 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.1@KRBTEST.COM'} at 127.16.197.1:40801
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
I20260807 11:22:54.898798 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:0
--local_ip_for_outbound_sockets=127.16.197.2
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:35715
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:39285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:22:55.238278 17988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:22:55.238582 17988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:22:55.238752 17988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:22:55.246371 17988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:22:55.246490 17988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:22:55.246707 17988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:22:55.257038 17988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:39285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:35715
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:22:55.258476 17988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:22:55.259995 17988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:22:55.273989 17996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:22:55.274139 17993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:22:55.274163 17994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:22:55.275427 17988 server_base.cc:1034] running on GCE node
I20260807 11:22:55.276132 17988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:22:55.277313 17988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:22:55.278525 17988 hybrid_clock.cc:648] HybridClock initialized: now 1786101775278483 us; error 31 us; skew 500 ppm
I20260807 11:22:55.278867 17988 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:22:55 dist-test-slave-2w3w krb5kdc[17737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101775, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:22:55.282071 17988 init.cc:377] Logged in from keytab as kudu/127.16.197.2@KRBTEST.COM (short username kudu)
I20260807 11:22:55.283596 17988 webserver.cc:460] Webserver started at http://127.16.197.2:36255/ using document root <none> and password file <none>
I20260807 11:22:55.284435 17988 fs_manager.cc:359] Metadata directory not provided
I20260807 11:22:55.284580 17988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:22:55.284957 17988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:22:55.288379 17988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-1/data/instance:
uuid: "95026fdbe1a5477799cfdba726c93536"
format_stamp: "Formatted at 2026-08-07 11:22:55 on dist-test-slave-2w3w"
server_key: "5ad2263db8abb1d942c23b86f0784fb3"
server_key_iv: "3333f76f25682def45d1d85e4cd5d737"
server_key_version: "encryptionkey@0"
I20260807 11:22:55.289191 17988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance:
uuid: "95026fdbe1a5477799cfdba726c93536"
format_stamp: "Formatted at 2026-08-07 11:22:55 on dist-test-slave-2w3w"
server_key: "5ad2263db8abb1d942c23b86f0784fb3"
server_key_iv: "3333f76f25682def45d1d85e4cd5d737"
server_key_version: "encryptionkey@0"
I20260807 11:22:55.294677 17988 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 11:22:55.298561 18003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:22:55.300130 17988 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:22:55.300341 17988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "95026fdbe1a5477799cfdba726c93536"
format_stamp: "Formatted at 2026-08-07 11:22:55 on dist-test-slave-2w3w"
server_key: "5ad2263db8abb1d942c23b86f0784fb3"
server_key_iv: "3333f76f25682def45d1d85e4cd5d737"
server_key_version: "encryptionkey@0"
I20260807 11:22:55.300581 17988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:22:55.315188 17988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:22:55.319454 17988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:22:55.319794 17988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:22:55.320863 17988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:22:55.322391 17988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:22:55.322494 17988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:22:55.322651 17988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:22:55.322731 17988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:22:55.357498 17988 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:42811
I20260807 11:22:55.357550 18125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:42811 every 8 connection(s)
I20260807 11:22:55.358897 17988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:22:55.368727 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 17988
I20260807 11:22:55.368930 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance
I20260807 11:22:55.369398 17172 external_mini_cluster.cc:1428] Setting key 70f80c1792819bf368e811acda526599
Aug 07 11:22:55 dist-test-slave-2w3w krb5kdc[17737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101775, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:22:55.380939 17984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:55.361518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:34669 (local address 127.16.197.62:35715)
0807 11:22:55.361695 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 11:22:55.361700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:22:55.362810 (+ 1110us) server_negotiation.cc:407] Connection header received
0807 11:22:55.364815 (+ 2005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:22:55.364820 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:22:55.364918 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:22:55.365035 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:22:55.366536 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:55.367354 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:55.368997 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:55.369465 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:55.369635 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:55.373443 (+ 3808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:22:55.373452 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:22:55.373460 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:22:55.373507 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:22:55.377462 (+ 3955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:55.378499 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:55.378502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:55.378505 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:55.378632 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:55.379189 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:55.379192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:55.379194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:55.379462 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:22:55.379574 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 11:22:55.380490 (+ 916us) server_negotiation.cc:299] Negotiation successful
0807 11:22:55.380667 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 11:22:55.382615 18128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:55.361914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:35715 (local address 127.16.197.2:34669)
0807 11:22:55.362614 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0807 11:22:55.362651 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 11:22:55.364521 (+ 1870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:22:55.365283 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:22:55.365292 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:22:55.365789 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:22:55.366354 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:22:55.366372 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:22:55.367501 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:22:55.367506 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:22:55.368766 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:22:55.368778 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:22:55.369724 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:22:55.369732 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:22:55.369978 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:55.370679 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:22:55.370702 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:22:55.373191 (+ 2489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:22:55.377621 (+ 4430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:22:55.377634 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:22:55.377654 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:22:55.378376 (+ 722us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:22:55.378752 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:22:55.378757 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:22:55.378760 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:22:55.379071 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:22:55.379590 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:22:55.379601 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:22:55.380096 (+ 495us) client_negotiation.cc:769] Sending connection context
0807 11:22:55.380398 (+ 302us) client_negotiation.cc:240] Negotiation successful
0807 11:22:55.381029 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":281,"threads_started":1}
I20260807 11:22:55.384421 18126 heartbeater.cc:347] Connected to a master server at 127.16.197.62:35715
I20260807 11:22:55.384764 18126 heartbeater.cc:464] Registering TS with master...
I20260807 11:22:55.385797 18126 heartbeater.cc:511] Master 127.16.197.62:35715 requested a full tablet report, sending...
I20260807 11:22:55.387490 17782 ts_manager.cc:194] Registered new tserver with Master: 95026fdbe1a5477799cfdba726c93536 (127.16.197.2:42811)
I20260807 11:22:55.388335 17782 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.2@KRBTEST.COM'} at 127.16.197.2:34669
WARNING: no policy specified for kudu/127.16.197.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
I20260807 11:22:55.417973 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:0
--local_ip_for_outbound_sockets=127.16.197.3
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:35715
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:39285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:22:55.818205 18131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:22:55.818584 18131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:22:55.818702 18131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:22:55.827284 18131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:22:55.827414 18131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:22:55.827605 18131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:22:55.838990 18131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:39285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:35715
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:22:55.840345 18131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:22:55.841779 18131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:22:55.855734 18136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:22:55.855780 18139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:22:55.855792 18137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:22:55.857211 18131 server_base.cc:1034] running on GCE node
I20260807 11:22:55.857879 18131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:22:55.858867 18131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:22:55.860090 18131 hybrid_clock.cc:648] HybridClock initialized: now 1786101775860042 us; error 39 us; skew 500 ppm
I20260807 11:22:55.860396 18131 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:22:55 dist-test-slave-2w3w krb5kdc[17737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101775, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:22:55.863667 18131 init.cc:377] Logged in from keytab as kudu/127.16.197.3@KRBTEST.COM (short username kudu)
I20260807 11:22:55.865168 18131 webserver.cc:460] Webserver started at http://127.16.197.3:44793/ using document root <none> and password file <none>
I20260807 11:22:55.865883 18131 fs_manager.cc:359] Metadata directory not provided
I20260807 11:22:55.866011 18131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:22:55.866350 18131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:22:55.869431 18131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-2/data/instance:
uuid: "f24b579abba14fceb877564ae46ccdd5"
format_stamp: "Formatted at 2026-08-07 11:22:55 on dist-test-slave-2w3w"
server_key: "a497dc90c020ac85a043b2b3f37e8a05"
server_key_iv: "0ef2e0fe5d07b020123fd1e1d0c215ee"
server_key_version: "encryptionkey@0"
I20260807 11:22:55.870240 18131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance:
uuid: "f24b579abba14fceb877564ae46ccdd5"
format_stamp: "Formatted at 2026-08-07 11:22:55 on dist-test-slave-2w3w"
server_key: "a497dc90c020ac85a043b2b3f37e8a05"
server_key_iv: "0ef2e0fe5d07b020123fd1e1d0c215ee"
server_key_version: "encryptionkey@0"
I20260807 11:22:55.875615 18131 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 11:22:55.879279 18146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:22:55.880589 18131 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 11:22:55.880720 18131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "f24b579abba14fceb877564ae46ccdd5"
format_stamp: "Formatted at 2026-08-07 11:22:55 on dist-test-slave-2w3w"
server_key: "a497dc90c020ac85a043b2b3f37e8a05"
server_key_iv: "0ef2e0fe5d07b020123fd1e1d0c215ee"
server_key_version: "encryptionkey@0"
I20260807 11:22:55.880887 18131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:22:55.881445 17982 heartbeater.cc:503] Master 127.16.197.62:35715 was elected leader, sending a full tablet report...
I20260807 11:22:55.900791 18131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:22:55.905298 18131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:22:55.905623 18131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:22:55.906636 18131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:22:55.908146 18131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:22:55.908224 18131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:22:55.908324 18131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:22:55.908371 18131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:22:55.943907 18131 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:43429
I20260807 11:22:55.943956 18264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:43429 every 8 connection(s)
I20260807 11:22:55.945252 18131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:22:55.950296 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 18131
I20260807 11:22:55.950498 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance
I20260807 11:22:55.951059 17172 external_mini_cluster.cc:1428] Setting key 8ebdf6baea0a86af8a699899d954a02f
Aug 07 11:22:55 dist-test-slave-2w3w krb5kdc[17737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101775, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:22:55.966235 18268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:55.947978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:44831 (local address 127.16.197.62:35715)
0807 11:22:55.948447 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0807 11:22:55.948452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:22:55.949370 (+ 918us) server_negotiation.cc:407] Connection header received
0807 11:22:55.951875 (+ 2505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:22:55.951880 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:22:55.951972 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:22:55.952100 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:22:55.953910 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:55.954844 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:55.956480 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:55.956887 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:55.956983 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:55.960452 (+ 3469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:22:55.960457 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:22:55.960460 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:22:55.960494 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:22:55.962902 (+ 2408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:55.963710 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:55.963713 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:55.963716 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:55.963865 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:55.964271 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:55.964275 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:55.964277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:55.964538 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:22:55.964728 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 11:22:55.965638 (+ 910us) server_negotiation.cc:299] Negotiation successful
0807 11:22:55.965853 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":266,"threads_started":1}
I20260807 11:22:55.967501 18269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:55.948437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:35715 (local address 127.16.197.3:44831)
0807 11:22:55.949128 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0807 11:22:55.949175 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 11:22:55.951582 (+ 2407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:22:55.952341 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:22:55.952349 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:22:55.953033 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:22:55.953715 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:22:55.953751 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:22:55.955067 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:22:55.955071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:22:55.956348 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:22:55.956361 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:22:55.957082 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:22:55.957086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:22:55.957283 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:55.957989 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:22:55.958017 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:22:55.960262 (+ 2245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:22:55.963092 (+ 2830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:22:55.963099 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:22:55.963112 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:22:55.963588 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:22:55.963972 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:22:55.963976 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:22:55.963978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:22:55.964165 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:22:55.964681 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:22:55.964689 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:22:55.965150 (+ 461us) client_negotiation.cc:769] Sending connection context
0807 11:22:55.965498 (+ 348us) client_negotiation.cc:240] Negotiation successful
0807 11:22:55.966149 (+ 651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":271,"threads_started":1}
I20260807 11:22:55.968813 18265 heartbeater.cc:347] Connected to a master server at 127.16.197.62:35715
I20260807 11:22:55.969121 18265 heartbeater.cc:464] Registering TS with master...
I20260807 11:22:55.969870 18265 heartbeater.cc:511] Master 127.16.197.62:35715 requested a full tablet report, sending...
I20260807 11:22:55.971486 17782 ts_manager.cc:194] Registered new tserver with Master: f24b579abba14fceb877564ae46ccdd5 (127.16.197.3:43429)
I20260807 11:22:55.972424 17782 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.3@KRBTEST.COM'} at 127.16.197.3:44831
I20260807 11:22:55.979485 17172 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 11:22:55.999590 18268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:55.986365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57104 (local address 127.16.197.62:35715)
0807 11:22:55.986557 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 11:22:55.986562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:22:55.987180 (+ 618us) server_negotiation.cc:407] Connection header received
0807 11:22:55.987320 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:22:55.987324 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:22:55.987395 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:22:55.987488 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:22:55.989014 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:55.989731 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:55.991013 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:55.991446 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:55.991548 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:55.994009 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:22:55.994013 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:22:55.994017 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:22:55.994048 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:22:55.996825 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:55.997608 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:55.997613 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:55.997616 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:55.997783 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:55.998179 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:55.998182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:55.998184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:55.998542 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:22:55.998799 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0807 11:22:55.999156 (+ 357us) server_negotiation.cc:299] Negotiation successful
0807 11:22:55.999348 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 11:22:56.006546 17782 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57104:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 11:22:56.014345 17782 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:22:56.051592 18281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:56.027958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:42811 (local address 127.0.0.1:44058)
0807 11:22:56.034952 (+ 6994us) negotiation.cc:107] Waiting for socket to connect
0807 11:22:56.035025 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0807 11:22:56.035285 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:22:56.043442 (+ 8157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:22:56.043449 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:22:56.043479 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:22:56.044073 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:22:56.044088 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.046351 (+ 2263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.046356 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:22:56.048422 (+ 2066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:22:56.048435 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.050434 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.050442 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:22:56.050657 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:56.050711 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 11:22:56.050857 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 11:22:56.051093 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6641,"mutex_wait_us":34,"spinlock_wait_cycles":11136,"thread_start_us":14892,"threads_started":1}
I20260807 11:22:56.052196 18279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:56.028067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44058 (local address 127.16.197.2:42811)
0807 11:22:56.042182 (+ 14115us) server_negotiation.cc:206] Beginning negotiation
0807 11:22:56.042191 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:22:56.042233 (+ 42us) server_negotiation.cc:407] Connection header received
0807 11:22:56.042355 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:22:56.042365 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:22:56.042679 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:22:56.043672 (+ 993us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:22:56.044499 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.046182 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.048601 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.050163 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.050335 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:56.050577 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 11:22:56.051677 (+ 1100us) server_negotiation.cc:299] Negotiation successful
0807 11:22:56.051865 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13951,"thread_start_us":13847,"threads_started":1}
I20260807 11:22:56.052269 18284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:56.029840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:44067 (local address 127.0.0.1:42236)
0807 11:22:56.044411 (+ 14571us) negotiation.cc:107] Waiting for socket to connect
0807 11:22:56.044432 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 11:22:56.044577 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:22:56.045156 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:22:56.045160 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:22:56.045186 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:22:56.045560 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:22:56.045569 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.047395 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.047400 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:22:56.048882 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:22:56.048895 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.051433 (+ 2538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.051440 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:22:56.051637 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:56.051679 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 11:22:56.051779 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 11:22:56.051900 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14418,"thread_start_us":14241,"threads_started":1}
I20260807 11:22:56.053009 18285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:56.030612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42236 (local address 127.16.197.1:44067)
0807 11:22:56.044060 (+ 13448us) server_negotiation.cc:206] Beginning negotiation
0807 11:22:56.044070 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 11:22:56.044615 (+ 545us) server_negotiation.cc:407] Connection header received
0807 11:22:56.044751 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:22:56.044758 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:22:56.044979 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:22:56.045166 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:22:56.045736 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.047224 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.049050 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.051193 (+ 2143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.051348 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:56.051559 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 11:22:56.052493 (+ 934us) server_negotiation.cc:299] Negotiation successful
0807 11:22:56.052689 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12638,"thread_start_us":12447,"threads_started":1}
I20260807 11:22:56.053009 18280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:56.028041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:43429 (local address 127.0.0.1:53372)
0807 11:22:56.042555 (+ 14514us) negotiation.cc:107] Waiting for socket to connect
0807 11:22:56.042576 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 11:22:56.042726 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:22:56.043469 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:22:56.043478 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:22:56.043520 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:22:56.044754 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:22:56.044767 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.046697 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.046710 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:22:56.050280 (+ 3570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:22:56.050297 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.052224 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.052234 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:22:56.052449 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:56.052481 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 11:22:56.052547 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 11:22:56.052673 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14394,"thread_start_us":778,"threads_started":1}
I20260807 11:22:56.053692 18282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:56.028626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53372 (local address 127.16.197.3:43429)
0807 11:22:56.042136 (+ 13510us) server_negotiation.cc:206] Beginning negotiation
0807 11:22:56.042150 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 11:22:56.042701 (+ 551us) server_negotiation.cc:407] Connection header received
0807 11:22:56.042917 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:22:56.042928 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:22:56.043181 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:22:56.044177 (+ 996us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:22:56.044949 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.046534 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.050470 (+ 3936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.051954 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.052786 (+ 832us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:56.053010 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 11:22:56.053090 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 11:22:56.053403 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13253,"thread_start_us":13538,"threads_started":1}
I20260807 11:22:56.056627 17913 tablet_service.cc:1467] Processing CreateTablet for tablet c952bad0777d41e299b986c62032adc4 (DEFAULT_TABLE table=test-table [id=c9fc98154c954a4688a8cd76fe45f0b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:22:56.056764 18053 tablet_service.cc:1467] Processing CreateTablet for tablet c952bad0777d41e299b986c62032adc4 (DEFAULT_TABLE table=test-table [id=c9fc98154c954a4688a8cd76fe45f0b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:22:56.056975 18197 tablet_service.cc:1467] Processing CreateTablet for tablet c952bad0777d41e299b986c62032adc4 (DEFAULT_TABLE table=test-table [id=c9fc98154c954a4688a8cd76fe45f0b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:22:56.057977 17913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c952bad0777d41e299b986c62032adc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:22:56.058298 18197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c952bad0777d41e299b986c62032adc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:22:56.058288 18053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c952bad0777d41e299b986c62032adc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:22:56.069284 18286 tablet_bootstrap.cc:492] T c952bad0777d41e299b986c62032adc4 P 6ff057bf28ac40c3bdd645ffe7dc1a21: Bootstrap starting.
I20260807 11:22:56.072842 18287 tablet_bootstrap.cc:492] T c952bad0777d41e299b986c62032adc4 P f24b579abba14fceb877564ae46ccdd5: Bootstrap starting.
I20260807 11:22:56.074188 18286 tablet_bootstrap.cc:654] T c952bad0777d41e299b986c62032adc4 P 6ff057bf28ac40c3bdd645ffe7dc1a21: Neither blocks nor log segments found. Creating new log.
I20260807 11:22:56.074597 18288 tablet_bootstrap.cc:492] T c952bad0777d41e299b986c62032adc4 P 95026fdbe1a5477799cfdba726c93536: Bootstrap starting.
I20260807 11:22:56.075639 18286 log.cc:824] T c952bad0777d41e299b986c62032adc4 P 6ff057bf28ac40c3bdd645ffe7dc1a21: Log is configured to *not* fsync() on all Append() calls
I20260807 11:22:56.077183 18287 tablet_bootstrap.cc:654] T c952bad0777d41e299b986c62032adc4 P f24b579abba14fceb877564ae46ccdd5: Neither blocks nor log segments found. Creating new log.
I20260807 11:22:56.078435 18288 tablet_bootstrap.cc:654] T c952bad0777d41e299b986c62032adc4 P 95026fdbe1a5477799cfdba726c93536: Neither blocks nor log segments found. Creating new log.
I20260807 11:22:56.079873 18287 log.cc:824] T c952bad0777d41e299b986c62032adc4 P f24b579abba14fceb877564ae46ccdd5: Log is configured to *not* fsync() on all Append() calls
I20260807 11:22:56.080943 18288 log.cc:824] T c952bad0777d41e299b986c62032adc4 P 95026fdbe1a5477799cfdba726c93536: Log is configured to *not* fsync() on all Append() calls
I20260807 11:22:56.087282 18286 tablet_bootstrap.cc:492] T c952bad0777d41e299b986c62032adc4 P 6ff057bf28ac40c3bdd645ffe7dc1a21: No bootstrap required, opened a new log
I20260807 11:22:56.087764 18286 ts_tablet_manager.cc:1397] T c952bad0777d41e299b986c62032adc4 P 6ff057bf28ac40c3bdd645ffe7dc1a21: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.009s
I20260807 11:22:56.092379 18288 tablet_bootstrap.cc:492] T c952bad0777d41e299b986c62032adc4 P 95026fdbe1a5477799cfdba726c93536: No bootstrap required, opened a new log
I20260807 11:22:56.092828 18288 ts_tablet_manager.cc:1397] T c952bad0777d41e299b986c62032adc4 P 95026fdbe1a5477799cfdba726c93536: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.011s
I20260807 11:22:56.092970 18287 tablet_bootstrap.cc:492] T c952bad0777d41e299b986c62032adc4 P f24b579abba14fceb877564ae46ccdd5: No bootstrap required, opened a new log
I20260807 11:22:56.093456 18287 ts_tablet_manager.cc:1397] T c952bad0777d41e299b986c62032adc4 P f24b579abba14fceb877564ae46ccdd5: Time spent bootstrapping tablet: real 0.021s user 0.015s sys 0.002s
I20260807 11:22:56.101534 18286 raft_consensus.cc:348] T c952bad0777d41e299b986c62032adc4 P 6ff057bf28ac40c3bdd645ffe7dc1a21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95026fdbe1a5477799cfdba726c93536" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 42811 } } peers { permanent_uuid: "f24b579abba14fceb877564ae46ccdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 43429 } } peers { permanent_uuid: "6ff057bf28ac40c3bdd645ffe7dc1a21" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 44067 } }
I20260807 11:22:56.102320 18286 raft_consensus.cc:374] T c952bad0777d41e299b986c62032adc4 P 6ff057bf28ac40c3bdd645ffe7dc1a21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:22:56.102466 18286 raft_consensus.cc:729] T c952bad0777d41e299b986c62032adc4 P 6ff057bf28ac40c3bdd645ffe7dc1a21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ff057bf28ac40c3bdd645ffe7dc1a21, State: Initialized, Role: FOLLOWER
I20260807 11:22:56.103296 18286 consensus_queue.cc:260] T c952bad0777d41e299b986c62032adc4 P 6ff057bf28ac40c3bdd645ffe7dc1a21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95026fdbe1a5477799cfdba726c93536" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 42811 } } peers { permanent_uuid: "f24b579abba14fceb877564ae46ccdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 43429 } } peers { permanent_uuid: "6ff057bf28ac40c3bdd645ffe7dc1a21" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 44067 } }
I20260807 11:22:56.103346 18288 raft_consensus.cc:348] T c952bad0777d41e299b986c62032adc4 P 95026fdbe1a5477799cfdba726c93536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95026fdbe1a5477799cfdba726c93536" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 42811 } } peers { permanent_uuid: "f24b579abba14fceb877564ae46ccdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 43429 } } peers { permanent_uuid: "6ff057bf28ac40c3bdd645ffe7dc1a21" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 44067 } }
I20260807 11:22:56.103929 18288 raft_consensus.cc:374] T c952bad0777d41e299b986c62032adc4 P 95026fdbe1a5477799cfdba726c93536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:22:56.104440 18288 raft_consensus.cc:729] T c952bad0777d41e299b986c62032adc4 P 95026fdbe1a5477799cfdba726c93536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95026fdbe1a5477799cfdba726c93536, State: Initialized, Role: FOLLOWER
I20260807 11:22:56.105429 18288 consensus_queue.cc:260] T c952bad0777d41e299b986c62032adc4 P 95026fdbe1a5477799cfdba726c93536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95026fdbe1a5477799cfdba726c93536" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 42811 } } peers { permanent_uuid: "f24b579abba14fceb877564ae46ccdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 43429 } } peers { permanent_uuid: "6ff057bf28ac40c3bdd645ffe7dc1a21" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 44067 } }
I20260807 11:22:56.105988 18286 ts_tablet_manager.cc:1428] T c952bad0777d41e299b986c62032adc4 P 6ff057bf28ac40c3bdd645ffe7dc1a21: Time spent starting tablet: real 0.018s user 0.012s sys 0.005s
I20260807 11:22:56.106899 18287 raft_consensus.cc:348] T c952bad0777d41e299b986c62032adc4 P f24b579abba14fceb877564ae46ccdd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95026fdbe1a5477799cfdba726c93536" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 42811 } } peers { permanent_uuid: "f24b579abba14fceb877564ae46ccdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 43429 } } peers { permanent_uuid: "6ff057bf28ac40c3bdd645ffe7dc1a21" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 44067 } }
I20260807 11:22:56.107563 18287 raft_consensus.cc:374] T c952bad0777d41e299b986c62032adc4 P f24b579abba14fceb877564ae46ccdd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:22:56.107756 18287 raft_consensus.cc:729] T c952bad0777d41e299b986c62032adc4 P f24b579abba14fceb877564ae46ccdd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f24b579abba14fceb877564ae46ccdd5, State: Initialized, Role: FOLLOWER
I20260807 11:22:56.107843 18126 heartbeater.cc:503] Master 127.16.197.62:35715 was elected leader, sending a full tablet report...
I20260807 11:22:56.108438 18287 consensus_queue.cc:260] T c952bad0777d41e299b986c62032adc4 P f24b579abba14fceb877564ae46ccdd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95026fdbe1a5477799cfdba726c93536" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 42811 } } peers { permanent_uuid: "f24b579abba14fceb877564ae46ccdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 43429 } } peers { permanent_uuid: "6ff057bf28ac40c3bdd645ffe7dc1a21" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 44067 } }
I20260807 11:22:56.109575 18288 ts_tablet_manager.cc:1428] T c952bad0777d41e299b986c62032adc4 P 95026fdbe1a5477799cfdba726c93536: Time spent starting tablet: real 0.016s user 0.013s sys 0.001s
W20260807 11:22:56.111004 18127 tablet.cc:2369] T c952bad0777d41e299b986c62032adc4 P 95026fdbe1a5477799cfdba726c93536: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:22:56.114295 18265 heartbeater.cc:503] Master 127.16.197.62:35715 was elected leader, sending a full tablet report...
I20260807 11:22:56.114943 18287 ts_tablet_manager.cc:1428] T c952bad0777d41e299b986c62032adc4 P f24b579abba14fceb877564ae46ccdd5: Time spent starting tablet: real 0.021s user 0.010s sys 0.010s
W20260807 11:22:56.197257 18267 tablet.cc:2369] T c952bad0777d41e299b986c62032adc4 P f24b579abba14fceb877564ae46ccdd5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:22:56.289932 18293 raft_consensus.cc:482] T c952bad0777d41e299b986c62032adc4 P 95026fdbe1a5477799cfdba726c93536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:22:56.290468 18293 raft_consensus.cc:504] T c952bad0777d41e299b986c62032adc4 P 95026fdbe1a5477799cfdba726c93536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95026fdbe1a5477799cfdba726c93536" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 42811 } } peers { permanent_uuid: "f24b579abba14fceb877564ae46ccdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 43429 } } peers { permanent_uuid: "6ff057bf28ac40c3bdd645ffe7dc1a21" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 44067 } }
I20260807 11:22:56.292634 18293 leader_election.cc:288] T c952bad0777d41e299b986c62032adc4 P 95026fdbe1a5477799cfdba726c93536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f24b579abba14fceb877564ae46ccdd5 (127.16.197.3:43429), 6ff057bf28ac40c3bdd645ffe7dc1a21 (127.16.197.1:44067)
I20260807 11:22:56.304280 18285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:56.294349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:40633 (local address 127.16.197.1:44067)
0807 11:22:56.294630 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0807 11:22:56.294636 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:22:56.294954 (+ 318us) server_negotiation.cc:407] Connection header received
0807 11:22:56.295290 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:22:56.295295 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:22:56.295375 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:22:56.295474 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:22:56.296222 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.297393 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.301151 (+ 3758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.302760 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.302978 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:56.303061 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 11:22:56.303790 (+ 729us) server_negotiation.cc:299] Negotiation successful
0807 11:22:56.303930 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260807 11:22:56.304332 18282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:56.293157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:57511 (local address 127.16.197.3:43429)
0807 11:22:56.293410 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 11:22:56.293414 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:22:56.293996 (+ 582us) server_negotiation.cc:407] Connection header received
0807 11:22:56.294387 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:22:56.294396 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:22:56.294502 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:22:56.294563 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:22:56.295755 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.297108 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.301151 (+ 4043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.302804 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.303002 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:56.303069 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 11:22:56.303820 (+ 751us) server_negotiation.cc:299] Negotiation successful
0807 11:22:56.303995 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 11:22:56.304672 18296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:56.293935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:44067 (local address 127.16.197.2:40633)
0807 11:22:56.294842 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0807 11:22:56.294858 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 11:22:56.295105 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:22:56.295497 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:22:56.295501 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:22:56.295527 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:22:56.296025 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:22:56.296034 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.297594 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.297600 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:22:56.300955 (+ 3355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:22:56.300985 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.303260 (+ 2275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.303267 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:22:56.303549 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:56.303648 (+ 99us) client_negotiation.cc:769] Sending connection context
0807 11:22:56.303744 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 11:22:56.303881 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"mutex_wait_us":79,"thread_start_us":541,"threads_started":1}
I20260807 11:22:56.304771 18295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:56.292815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:43429 (local address 127.16.197.2:57511)
0807 11:22:56.293883 (+ 1068us) negotiation.cc:107] Waiting for socket to connect
0807 11:22:56.293903 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 11:22:56.294146 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:22:56.294796 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:22:56.294799 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:22:56.294820 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:22:56.295601 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:22:56.295615 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.297261 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.297265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:22:56.300992 (+ 3727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:22:56.301001 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.303243 (+ 2242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.303251 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:22:56.303556 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:56.303645 (+ 89us) client_negotiation.cc:769] Sending connection context
0807 11:22:56.303778 (+ 133us) client_negotiation.cc:240] Negotiation successful
0807 11:22:56.303883 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":910,"mutex_wait_us":95,"thread_start_us":671,"threads_started":1}
I20260807 11:22:56.306155 18217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c952bad0777d41e299b986c62032adc4" candidate_uuid: "95026fdbe1a5477799cfdba726c93536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f24b579abba14fceb877564ae46ccdd5" is_pre_election: true
I20260807 11:22:56.306166 17935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c952bad0777d41e299b986c62032adc4" candidate_uuid: "95026fdbe1a5477799cfdba726c93536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ff057bf28ac40c3bdd645ffe7dc1a21" is_pre_election: true
I20260807 11:22:56.306799 17935 raft_consensus.cc:2454] T c952bad0777d41e299b986c62032adc4 P 6ff057bf28ac40c3bdd645ffe7dc1a21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95026fdbe1a5477799cfdba726c93536 in term 0.
I20260807 11:22:56.306809 18217 raft_consensus.cc:2454] T c952bad0777d41e299b986c62032adc4 P f24b579abba14fceb877564ae46ccdd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95026fdbe1a5477799cfdba726c93536 in term 0.
I20260807 11:22:56.307619 18005 leader_election.cc:302] T c952bad0777d41e299b986c62032adc4 P 95026fdbe1a5477799cfdba726c93536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ff057bf28ac40c3bdd645ffe7dc1a21, 95026fdbe1a5477799cfdba726c93536; no voters:
I20260807 11:22:56.308074 18293 raft_consensus.cc:2781] T c952bad0777d41e299b986c62032adc4 P 95026fdbe1a5477799cfdba726c93536 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:22:56.308312 18293 raft_consensus.cc:482] T c952bad0777d41e299b986c62032adc4 P 95026fdbe1a5477799cfdba726c93536 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:22:56.308492 18293 raft_consensus.cc:3037] T c952bad0777d41e299b986c62032adc4 P 95026fdbe1a5477799cfdba726c93536 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:22:56.311093 18293 raft_consensus.cc:504] T c952bad0777d41e299b986c62032adc4 P 95026fdbe1a5477799cfdba726c93536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95026fdbe1a5477799cfdba726c93536" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 42811 } } peers { permanent_uuid: "f24b579abba14fceb877564ae46ccdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 43429 } } peers { permanent_uuid: "6ff057bf28ac40c3bdd645ffe7dc1a21" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 44067 } }
I20260807 11:22:56.311791 18293 leader_election.cc:288] T c952bad0777d41e299b986c62032adc4 P 95026fdbe1a5477799cfdba726c93536 [CANDIDATE]: Term 1 election: Requested vote from peers f24b579abba14fceb877564ae46ccdd5 (127.16.197.3:43429), 6ff057bf28ac40c3bdd645ffe7dc1a21 (127.16.197.1:44067)
I20260807 11:22:56.312263 18217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c952bad0777d41e299b986c62032adc4" candidate_uuid: "95026fdbe1a5477799cfdba726c93536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f24b579abba14fceb877564ae46ccdd5"
I20260807 11:22:56.312358 17935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c952bad0777d41e299b986c62032adc4" candidate_uuid: "95026fdbe1a5477799cfdba726c93536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ff057bf28ac40c3bdd645ffe7dc1a21"
I20260807 11:22:56.312503 18217 raft_consensus.cc:3037] T c952bad0777d41e299b986c62032adc4 P f24b579abba14fceb877564ae46ccdd5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:22:56.312621 17935 raft_consensus.cc:3037] T c952bad0777d41e299b986c62032adc4 P 6ff057bf28ac40c3bdd645ffe7dc1a21 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:22:56.315263 18217 raft_consensus.cc:2454] T c952bad0777d41e299b986c62032adc4 P f24b579abba14fceb877564ae46ccdd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95026fdbe1a5477799cfdba726c93536 in term 1.
I20260807 11:22:56.315604 17935 raft_consensus.cc:2454] T c952bad0777d41e299b986c62032adc4 P 6ff057bf28ac40c3bdd645ffe7dc1a21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95026fdbe1a5477799cfdba726c93536 in term 1.
I20260807 11:22:56.315802 18005 leader_election.cc:302] T c952bad0777d41e299b986c62032adc4 P 95026fdbe1a5477799cfdba726c93536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95026fdbe1a5477799cfdba726c93536, f24b579abba14fceb877564ae46ccdd5; no voters:
I20260807 11:22:56.316310 18293 raft_consensus.cc:2781] T c952bad0777d41e299b986c62032adc4 P 95026fdbe1a5477799cfdba726c93536 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:22:56.317144 18293 raft_consensus.cc:686] T c952bad0777d41e299b986c62032adc4 P 95026fdbe1a5477799cfdba726c93536 [term 1 LEADER]: Becoming Leader. State: Replica: 95026fdbe1a5477799cfdba726c93536, State: Running, Role: LEADER
I20260807 11:22:56.317659 18293 consensus_queue.cc:237] T c952bad0777d41e299b986c62032adc4 P 95026fdbe1a5477799cfdba726c93536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95026fdbe1a5477799cfdba726c93536" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 42811 } } peers { permanent_uuid: "f24b579abba14fceb877564ae46ccdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 43429 } } peers { permanent_uuid: "6ff057bf28ac40c3bdd645ffe7dc1a21" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 44067 } }
I20260807 11:22:56.323580 17781 catalog_manager.cc:5495] T c952bad0777d41e299b986c62032adc4 P 95026fdbe1a5477799cfdba726c93536 reported cstate change: term changed from 0 to 1, leader changed from <none> to 95026fdbe1a5477799cfdba726c93536 (127.16.197.2). New cstate: current_term: 1 leader_uuid: "95026fdbe1a5477799cfdba726c93536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95026fdbe1a5477799cfdba726c93536" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 42811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f24b579abba14fceb877564ae46ccdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 43429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ff057bf28ac40c3bdd645ffe7dc1a21" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 44067 } health_report { overall_health: UNKNOWN } } }
W20260807 11:22:56.350561 17983 tablet.cc:2369] T c952bad0777d41e299b986c62032adc4 P 6ff057bf28ac40c3bdd645ffe7dc1a21: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 11:22:56 dist-test-slave-2w3w krb5kdc[17737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.1@KRBTEST.COM
I20260807 11:22:56.369275 18285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:56.348363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42244 (local address 127.16.197.1:44067)
0807 11:22:56.348549 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 11:22:56.348554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:22:56.349562 (+ 1008us) server_negotiation.cc:407] Connection header received
0807 11:22:56.349796 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:22:56.349800 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:22:56.349907 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:22:56.350042 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:22:56.352746 (+ 2704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.353554 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.355534 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.356221 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.356375 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:56.361827 (+ 5452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:22:56.361840 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:22:56.361854 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:22:56.361894 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:22:56.365776 (+ 3882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:56.366813 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:56.366820 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:56.366822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:56.367044 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:56.367455 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:56.367458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:56.367460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:56.367869 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:22:56.368131 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0807 11:22:56.368547 (+ 416us) server_negotiation.cc:299] Negotiation successful
0807 11:22:56.368713 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
Aug 07 11:22:56 dist-test-slave-2w3w krb5kdc[17737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101776, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 11:22:56 dist-test-slave-2w3w krb5kdc[17737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101776, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.197.1@KRBTEST.COM
I20260807 11:22:56.421972 18285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:56.406389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42252 (local address 127.16.197.1:44067)
0807 11:22:56.406642 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 11:22:56.406651 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:22:56.407035 (+ 384us) server_negotiation.cc:407] Connection header received
0807 11:22:56.407287 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:22:56.407291 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:22:56.407359 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:22:56.407471 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:22:56.408937 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.409946 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.411273 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.411863 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.412012 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:56.416285 (+ 4273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:22:56.416296 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:22:56.416300 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:22:56.416336 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:22:56.418921 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:56.419613 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:56.419619 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:56.419624 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:56.419779 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:56.420215 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:56.420219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:56.420221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:56.420536 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:22:56.420932 (+ 396us) server_negotiation.cc:1035] Waiting for connection context
0807 11:22:56.421212 (+ 280us) server_negotiation.cc:299] Negotiation successful
0807 11:22:56.421494 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 11:22:56 dist-test-slave-2w3w krb5kdc[17737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 11:22:56 dist-test-slave-2w3w krb5kdc[17737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.1@KRBTEST.COM
I20260807 11:22:56.471159 18285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:56.456693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42268 (local address 127.16.197.1:44067)
0807 11:22:56.456973 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0807 11:22:56.456980 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:22:56.457267 (+ 287us) server_negotiation.cc:407] Connection header received
0807 11:22:56.457465 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:22:56.457469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:22:56.457540 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:22:56.457638 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:22:56.459079 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.459887 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.461182 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.461641 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.461783 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:56.465899 (+ 4116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:22:56.465904 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:22:56.465908 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:22:56.465947 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:22:56.468546 (+ 2599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:56.469196 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:56.469200 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:56.469203 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:56.469334 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:56.469743 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:56.469746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:56.469748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:56.470044 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:22:56.470312 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0807 11:22:56.470503 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 11:22:56.470710 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
W20260807 11:22:56.472985 17893 tablet_service.cc:2908] Rejecting scan request for tablet c952bad0777d41e299b986c62032adc4: Uninitialized: clean time has not yet been initialized
I20260807 11:22:56.476233 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 17848
I20260807 11:22:56.491127 18285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:56.477570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42274 (local address 127.16.197.1:44067)
0807 11:22:56.477756 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 11:22:56.477761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:22:56.478108 (+ 347us) server_negotiation.cc:407] Connection header received
0807 11:22:56.478251 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:22:56.478254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:22:56.478325 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:22:56.478404 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:22:56.479656 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.480389 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.481562 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.481975 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.482103 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:56.484522 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:22:56.484527 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:22:56.484530 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:22:56.484563 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:22:56.488196 (+ 3633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:56.488816 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:56.488819 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:56.488821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:56.489026 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:56.489462 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:56.489468 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:56.489471 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:56.489851 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:22:56.490261 (+ 410us) server_negotiation.cc:1035] Waiting for connection context
0807 11:22:56.490464 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 11:22:56.490710 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 11:22:56.491778 17977 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:22:56.584249 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 17848
I20260807 11:22:56.605814 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 17988
Aug 07 11:22:56 dist-test-slave-2w3w krb5kdc[17737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.2@KRBTEST.COM
I20260807 11:22:56.622514 18312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:56.606706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44060 (local address 127.16.197.2:42811)
0807 11:22:56.607245 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0807 11:22:56.607249 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:22:56.607274 (+ 25us) server_negotiation.cc:407] Connection header received
0807 11:22:56.607340 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:22:56.607344 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:22:56.607408 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:22:56.607506 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:22:56.608860 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.609560 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.610943 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.611579 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.611684 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:56.615934 (+ 4250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:22:56.615947 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:22:56.615963 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:22:56.616014 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:22:56.619361 (+ 3347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:56.620181 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:56.620189 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:56.620192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:56.620345 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:56.620771 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:56.620776 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:56.620779 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:56.621165 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:22:56.621519 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0807 11:22:56.621854 (+ 335us) server_negotiation.cc:299] Negotiation successful
0807 11:22:56.622110 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":319,"threads_started":1}
I20260807 11:22:56.623366 18121 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:22:56.718606 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 17988
I20260807 11:22:56.738396 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 18131
Aug 07 11:22:56 dist-test-slave-2w3w krb5kdc[17737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.3@KRBTEST.COM
I20260807 11:22:56.754067 18282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:56.739280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53384 (local address 127.16.197.3:43429)
0807 11:22:56.739476 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 11:22:56.739481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:22:56.739500 (+ 19us) server_negotiation.cc:407] Connection header received
0807 11:22:56.739644 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:22:56.739648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:22:56.739716 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:22:56.739818 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:22:56.741185 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.741936 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.743442 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:56.743859 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:56.743961 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:56.748166 (+ 4205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:22:56.748183 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:22:56.748199 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:22:56.748236 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:22:56.751229 (+ 2993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:56.751979 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:56.751987 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:56.751989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:56.752123 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:56.752469 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:56.752472 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:56.752474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:56.752845 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:22:56.753069 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 11:22:56.753480 (+ 411us) server_negotiation.cc:299] Negotiation successful
0807 11:22:56.753702 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 11:22:56.754922 18260 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:22:56.848292 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 18131
I20260807 11:22:56.867969 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 17750
I20260807 11:22:56.869370 17817 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:22:56.983700 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 17750
2026-08-07T11:22:57Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4506 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:22:57 dist-test-slave-2w3w krb5kdc[18318](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:22:57 dist-test-slave-2w3w krb5kdc[18318](info): set up 2 sockets
Aug 07 11:22:57 dist-test-slave-2w3w krb5kdc[18318](info): commencing operation
krb5kdc: starting...
W20260807 11:22:58.125026 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.005s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:22:58 dist-test-slave-2w3w krb5kdc[18318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:22:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:22:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:22:58.326463 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:43575
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:44901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:43575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:22:58.653589 18333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:22:58.653963 18333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:22:58.654131 18333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:22:58.661543 18333 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:22:58.661700 18333 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 11:22:58.661811 18333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:22:58.661929 18333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:22:58.662015 18333 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:22:58.662102 18333 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:22:58.672236 18333 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:44901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:43575
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:43575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:22:58.673616 18333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:22:58.675094 18333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:22:58.687405 18338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:22:58.687353 18339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:22:58.687353 18341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:22:58.688555 18333 server_base.cc:1034] running on GCE node
I20260807 11:22:58.689391 18333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:22:58.690901 18333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:22:58.692196 18333 hybrid_clock.cc:648] HybridClock initialized: now 1786101778692115 us; error 54 us; skew 500 ppm
I20260807 11:22:58.692554 18333 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:22:58 dist-test-slave-2w3w krb5kdc[18318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101778, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:22:58.695971 18333 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:22:58.697500 18333 webserver.cc:460] Webserver started at http://127.16.197.62:40305/ using document root <none> and password file <none>
I20260807 11:22:58.698258 18333 fs_manager.cc:359] Metadata directory not provided
I20260807 11:22:58.698396 18333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:22:58.698765 18333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:22:58.701912 18333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "b527b68d53bf4086bc4f8b2f7627b6c5"
format_stamp: "Formatted at 2026-08-07 11:22:58 on dist-test-slave-2w3w"
server_key: "ad0f5bcd7356dc4537b76c2e8e6b51a2"
server_key_iv: "1c6dd91e067ab8770cbde5acb1aac672"
server_key_version: "encryptionkey@0"
I20260807 11:22:58.702725 18333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "b527b68d53bf4086bc4f8b2f7627b6c5"
format_stamp: "Formatted at 2026-08-07 11:22:58 on dist-test-slave-2w3w"
server_key: "ad0f5bcd7356dc4537b76c2e8e6b51a2"
server_key_iv: "1c6dd91e067ab8770cbde5acb1aac672"
server_key_version: "encryptionkey@0"
I20260807 11:22:58.708330 18333 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 11:22:58.711858 18349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:22:58.713524 18333 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260807 11:22:58.713788 18333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "b527b68d53bf4086bc4f8b2f7627b6c5"
format_stamp: "Formatted at 2026-08-07 11:22:58 on dist-test-slave-2w3w"
server_key: "ad0f5bcd7356dc4537b76c2e8e6b51a2"
server_key_iv: "1c6dd91e067ab8770cbde5acb1aac672"
server_key_version: "encryptionkey@0"
I20260807 11:22:58.714037 18333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:22:58.744727 18333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:22:58.749403 18333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:22:58.749714 18333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:22:58.772598 18333 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:43575
I20260807 11:22:58.772620 18416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:43575 every 8 connection(s)
I20260807 11:22:58.774060 18333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:22:58.776332 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 18333
I20260807 11:22:58.776579 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:22:58.777235 17172 external_mini_cluster.cc:1428] Setting key 872571e7597cf66f1d9d4604a4417b88
I20260807 11:22:58.778883 18417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 11:22:58 dist-test-slave-2w3w krb5kdc[18318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:22:58.794106 18417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b527b68d53bf4086bc4f8b2f7627b6c5: Bootstrap starting.
I20260807 11:22:58.797770 18417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b527b68d53bf4086bc4f8b2f7627b6c5: Neither blocks nor log segments found. Creating new log.
I20260807 11:22:58.799160 18417 log.cc:824] T 00000000000000000000000000000000 P b527b68d53bf4086bc4f8b2f7627b6c5: Log is configured to *not* fsync() on all Append() calls
I20260807 11:22:58.802031 18420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:58.779366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46052 (local address 127.16.197.62:43575)
0807 11:22:58.780285 (+ 919us) server_negotiation.cc:206] Beginning negotiation
0807 11:22:58.780305 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 11:22:58.780367 (+ 62us) server_negotiation.cc:407] Connection header received
0807 11:22:58.782695 (+ 2328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:22:58.782726 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:22:58.783444 (+ 718us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:22:58.783938 (+ 494us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:22:58.785226 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:58.787371 (+ 2145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:58.788635 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:58.789671 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:58.789861 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:58.794544 (+ 4683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:22:58.794552 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:22:58.794568 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:22:58.794609 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:22:58.797653 (+ 3044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:58.798519 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:58.798525 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:58.798532 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:58.798703 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:58.799173 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:58.799176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:58.799178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:58.799748 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:22:58.800068 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0807 11:22:58.800607 (+ 539us) server_negotiation.cc:299] Negotiation successful
0807 11:22:58.801025 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":367,"threads_started":1}
I20260807 11:22:58.802264 18417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b527b68d53bf4086bc4f8b2f7627b6c5: No bootstrap required, opened a new log
I20260807 11:22:58.811587 18417 raft_consensus.cc:348] T 00000000000000000000000000000000 P b527b68d53bf4086bc4f8b2f7627b6c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b527b68d53bf4086bc4f8b2f7627b6c5" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 43575 } }
I20260807 11:22:58.812013 18417 raft_consensus.cc:374] T 00000000000000000000000000000000 P b527b68d53bf4086bc4f8b2f7627b6c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:22:58.812142 18417 raft_consensus.cc:729] T 00000000000000000000000000000000 P b527b68d53bf4086bc4f8b2f7627b6c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b527b68d53bf4086bc4f8b2f7627b6c5, State: Initialized, Role: FOLLOWER
I20260807 11:22:58.812865 18417 consensus_queue.cc:260] T 00000000000000000000000000000000 P b527b68d53bf4086bc4f8b2f7627b6c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b527b68d53bf4086bc4f8b2f7627b6c5" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 43575 } }
I20260807 11:22:58.813082 18417 raft_consensus.cc:388] T 00000000000000000000000000000000 P b527b68d53bf4086bc4f8b2f7627b6c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:22:58.813232 18417 raft_consensus.cc:482] T 00000000000000000000000000000000 P b527b68d53bf4086bc4f8b2f7627b6c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:22:58.813428 18417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b527b68d53bf4086bc4f8b2f7627b6c5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:22:58.815948 18417 raft_consensus.cc:504] T 00000000000000000000000000000000 P b527b68d53bf4086bc4f8b2f7627b6c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b527b68d53bf4086bc4f8b2f7627b6c5" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 43575 } }
I20260807 11:22:58.816484 18417 leader_election.cc:302] T 00000000000000000000000000000000 P b527b68d53bf4086bc4f8b2f7627b6c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b527b68d53bf4086bc4f8b2f7627b6c5; no voters:
I20260807 11:22:58.817104 18417 leader_election.cc:288] T 00000000000000000000000000000000 P b527b68d53bf4086bc4f8b2f7627b6c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:22:58.817287 18422 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b527b68d53bf4086bc4f8b2f7627b6c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:22:58.818207 18422 raft_consensus.cc:686] T 00000000000000000000000000000000 P b527b68d53bf4086bc4f8b2f7627b6c5 [term 1 LEADER]: Becoming Leader. State: Replica: b527b68d53bf4086bc4f8b2f7627b6c5, State: Running, Role: LEADER
I20260807 11:22:58.818818 18422 consensus_queue.cc:237] T 00000000000000000000000000000000 P b527b68d53bf4086bc4f8b2f7627b6c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b527b68d53bf4086bc4f8b2f7627b6c5" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 43575 } }
I20260807 11:22:58.819895 18417 sys_catalog.cc:564] T 00000000000000000000000000000000 P b527b68d53bf4086bc4f8b2f7627b6c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:22:58.822247 18424 sys_catalog.cc:455] T 00000000000000000000000000000000 P b527b68d53bf4086bc4f8b2f7627b6c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b527b68d53bf4086bc4f8b2f7627b6c5. Latest consensus state: current_term: 1 leader_uuid: "b527b68d53bf4086bc4f8b2f7627b6c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b527b68d53bf4086bc4f8b2f7627b6c5" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 43575 } } }
I20260807 11:22:58.822216 18423 sys_catalog.cc:455] T 00000000000000000000000000000000 P b527b68d53bf4086bc4f8b2f7627b6c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b527b68d53bf4086bc4f8b2f7627b6c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b527b68d53bf4086bc4f8b2f7627b6c5" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 43575 } } }
I20260807 11:22:58.822690 18424 sys_catalog.cc:458] T 00000000000000000000000000000000 P b527b68d53bf4086bc4f8b2f7627b6c5 [sys.catalog]: This master's current role is: LEADER
I20260807 11:22:58.822700 18423 sys_catalog.cc:458] T 00000000000000000000000000000000 P b527b68d53bf4086bc4f8b2f7627b6c5 [sys.catalog]: This master's current role is: LEADER
I20260807 11:22:58.824641 18429 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:22:58.831429 18429 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:22:58.841416 18429 catalog_manager.cc:1347] Generated new cluster ID: 81f470c1d5514fd2a1e97fae05a8398f
I20260807 11:22:58.841549 18429 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:22:58.857504 18429 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:22:58.859489 18429 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:22:58.869169 18429 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b527b68d53bf4086bc4f8b2f7627b6c5: Generated new TSK 0
I20260807 11:22:58.870118 18429 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.197.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
I20260807 11:22:58.938797 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:0
--local_ip_for_outbound_sockets=127.16.197.1
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:43575
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:44901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:22:59.326042 18444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:22:59.326370 18444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:22:59.326555 18444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:22:59.334267 18444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:22:59.334432 18444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:22:59.334594 18444 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 11:22:59.334765 18444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:22:59.345098 18444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:44901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.197.62:43575
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:22:59.346457 18444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:22:59.347934 18444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:22:59.366432 18451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:22:59.371780 18450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:22:59.374557 18453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:22:59.374938 18444 server_base.cc:1034] running on GCE node
I20260807 11:22:59.375907 18444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:22:59.376971 18444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:22:59.378268 18444 hybrid_clock.cc:648] HybridClock initialized: now 1786101779378191 us; error 56 us; skew 500 ppm
I20260807 11:22:59.378598 18444 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:22:59 dist-test-slave-2w3w krb5kdc[18318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101779, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:22:59.381981 18444 init.cc:377] Logged in from keytab as kudu/127.16.197.1@KRBTEST.COM (short username kudu)
I20260807 11:22:59.387372 18444 webserver.cc:460] Webserver started at http://127.16.197.1:35157/ using document root <none> and password file <none>
I20260807 11:22:59.388257 18444 fs_manager.cc:359] Metadata directory not provided
I20260807 11:22:59.388393 18444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:22:59.388758 18444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:22:59.391921 18444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-0/data/instance:
uuid: "257609186abd4d8eb658659a01aa71ab"
format_stamp: "Formatted at 2026-08-07 11:22:59 on dist-test-slave-2w3w"
server_key: "aef67b2dc32ce9018494b957160976cb"
server_key_iv: "53da1b819d52116abc4724971124b888"
server_key_version: "encryptionkey@0"
I20260807 11:22:59.392752 18444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance:
uuid: "257609186abd4d8eb658659a01aa71ab"
format_stamp: "Formatted at 2026-08-07 11:22:59 on dist-test-slave-2w3w"
server_key: "aef67b2dc32ce9018494b957160976cb"
server_key_iv: "53da1b819d52116abc4724971124b888"
server_key_version: "encryptionkey@0"
I20260807 11:22:59.398181 18444 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 11:22:59.401873 18461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:22:59.403370 18444 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 11:22:59.403553 18444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "257609186abd4d8eb658659a01aa71ab"
format_stamp: "Formatted at 2026-08-07 11:22:59 on dist-test-slave-2w3w"
server_key: "aef67b2dc32ce9018494b957160976cb"
server_key_iv: "53da1b819d52116abc4724971124b888"
server_key_version: "encryptionkey@0"
I20260807 11:22:59.403776 18444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:22:59.419337 18444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:22:59.424094 18444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:22:59.424396 18444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:22:59.440109 18444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:22:59.440330 18444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260807 11:22:59.440555 18444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:22:59.440672 18444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 11:22:59 dist-test-slave-2w3w krb5kdc[18318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101779, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:22:59.480355 18469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:59.429317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:43575 (local address 127.16.197.1:53879)
0807 11:22:59.446760 (+ 17443us) negotiation.cc:107] Waiting for socket to connect
0807 11:22:59.446835 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0807 11:22:59.450343 (+ 3508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:22:59.451662 (+ 1319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:22:59.451677 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:22:59.452829 (+ 1152us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:22:59.453964 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:22:59.453995 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:22:59.456200 (+ 2205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:22:59.456209 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:22:59.459361 (+ 3152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:22:59.459414 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:22:59.461108 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:22:59.461117 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:22:59.461516 (+ 399us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:59.462515 (+ 999us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:22:59.462557 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:22:59.466901 (+ 4344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:22:59.473478 (+ 6577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:22:59.473489 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:22:59.473524 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:22:59.474934 (+ 1410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:22:59.475706 (+ 772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:22:59.475714 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:22:59.475721 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:22:59.476129 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:22:59.477043 (+ 914us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:22:59.477059 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:22:59.477714 (+ 655us) client_negotiation.cc:769] Sending connection context
0807 11:22:59.478139 (+ 425us) client_negotiation.cc:240] Negotiation successful
0807 11:22:59.478915 (+ 776us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17046,"thread_start_us":16753,"threads_started":1}
I20260807 11:22:59.482077 18470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:59.430045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:53879 (local address 127.16.197.62:43575)
0807 11:22:59.446076 (+ 16031us) server_negotiation.cc:206] Beginning negotiation
0807 11:22:59.446087 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 11:22:59.447323 (+ 1236us) server_negotiation.cc:407] Connection header received
0807 11:22:59.450771 (+ 3448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:22:59.450780 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:22:59.450890 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:22:59.451015 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:22:59.454272 (+ 3257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:59.455919 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:59.459613 (+ 3694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:59.460509 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:59.460642 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:59.467250 (+ 6608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:22:59.467258 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:22:59.467265 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:22:59.467324 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:22:59.473102 (+ 5778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:59.475258 (+ 2156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:59.475264 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:59.475279 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:59.475515 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:59.476324 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:59.476330 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:59.476333 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:59.476767 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:22:59.476932 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 11:22:59.481244 (+ 4312us) server_negotiation.cc:299] Negotiation successful
0807 11:22:59.481566 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15857,"thread_start_us":16035,"threads_started":1}
I20260807 11:22:59.498596 18444 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:35649
I20260807 11:22:59.498597 18587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:35649 every 8 connection(s)
I20260807 11:22:59.501015 18444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:22:59.502993 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 18444
I20260807 11:22:59.503235 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance
I20260807 11:22:59.503887 17172 external_mini_cluster.cc:1428] Setting key 84dc5107e906c32baebe937d3c235ce1
I20260807 11:22:59.505491 18588 heartbeater.cc:347] Connected to a master server at 127.16.197.62:43575
I20260807 11:22:59.506949 18588 heartbeater.cc:464] Registering TS with master...
I20260807 11:22:59.508126 18588 heartbeater.cc:511] Master 127.16.197.62:43575 requested a full tablet report, sending...
I20260807 11:22:59.509657 18577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:59.493230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:43575 (local address 127.16.197.1:34741)
0807 11:22:59.494036 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0807 11:22:59.494062 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 11:22:59.494232 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:22:59.494929 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:22:59.494934 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:22:59.495784 (+ 850us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:22:59.496434 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:22:59.496447 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:22:59.497814 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:22:59.497821 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:22:59.499406 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:22:59.499420 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:22:59.500258 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:22:59.500265 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:22:59.500456 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:59.501161 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:22:59.501184 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:22:59.502216 (+ 1032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:22:59.505993 (+ 3777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:22:59.505998 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:22:59.506002 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:22:59.506386 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:22:59.507327 (+ 941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:22:59.507333 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:22:59.507336 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:22:59.507539 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:22:59.508556 (+ 1017us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:22:59.508562 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:22:59.508797 (+ 235us) client_negotiation.cc:769] Sending connection context
0807 11:22:59.508946 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 11:22:59.509314 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":476,"threads_started":1}
I20260807 11:22:59.510653 18470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:59.493894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:34741 (local address 127.16.197.62:43575)
0807 11:22:59.494145 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 11:22:59.494154 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:22:59.494182 (+ 28us) server_negotiation.cc:407] Connection header received
0807 11:22:59.494455 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:22:59.494464 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:22:59.494567 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:22:59.494652 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:22:59.496594 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:59.497632 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:59.499562 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:59.500050 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:59.500192 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:59.502454 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:22:59.502463 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:22:59.502471 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:22:59.502522 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:22:59.505857 (+ 3335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:59.506904 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:59.506910 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:59.506914 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:59.507139 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:59.507855 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:59.507861 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:59.507868 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:59.508391 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:22:59.509976 (+ 1585us) server_negotiation.cc:1035] Waiting for connection context
0807 11:22:59.510134 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 11:22:59.510398 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260807 11:22:59.515452 18367 ts_manager.cc:194] Registered new tserver with Master: 257609186abd4d8eb658659a01aa71ab (127.16.197.1:35649)
I20260807 11:22:59.517163 18367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.1@KRBTEST.COM'} at 127.16.197.1:53879
WARNING: no policy specified for kudu/127.16.197.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
I20260807 11:22:59.556468 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:0
--local_ip_for_outbound_sockets=127.16.197.2
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:43575
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:44901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:22:59.888025 18592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:22:59.888309 18592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:22:59.888485 18592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:22:59.896062 18592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:22:59.896180 18592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:22:59.896365 18592 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 11:22:59.896514 18592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:22:59.906797 18592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:44901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.197.62:43575
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:22:59.908449 18592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:22:59.909874 18592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:22:59.923820 18598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:22:59.924409 18597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:22:59.924010 18600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:22:59.925300 18592 server_base.cc:1034] running on GCE node
I20260807 11:22:59.925992 18592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:22:59.927023 18592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:22:59.928309 18592 hybrid_clock.cc:648] HybridClock initialized: now 1786101779928241 us; error 38 us; skew 500 ppm
I20260807 11:22:59.928627 18592 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:22:59 dist-test-slave-2w3w krb5kdc[18318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101779, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:22:59.932017 18592 init.cc:377] Logged in from keytab as kudu/127.16.197.2@KRBTEST.COM (short username kudu)
I20260807 11:22:59.933640 18592 webserver.cc:460] Webserver started at http://127.16.197.2:40405/ using document root <none> and password file <none>
I20260807 11:22:59.934396 18592 fs_manager.cc:359] Metadata directory not provided
I20260807 11:22:59.934553 18592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:22:59.934919 18592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:22:59.937994 18592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-1/data/instance:
uuid: "88ec7cf9488d40af9c742bd902fa595b"
format_stamp: "Formatted at 2026-08-07 11:22:59 on dist-test-slave-2w3w"
server_key: "a0ffcc3b44befafc4af956d0aba0a338"
server_key_iv: "d8b32f81d4eeba74a43c256a3cf6a86b"
server_key_version: "encryptionkey@0"
I20260807 11:22:59.938862 18592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance:
uuid: "88ec7cf9488d40af9c742bd902fa595b"
format_stamp: "Formatted at 2026-08-07 11:22:59 on dist-test-slave-2w3w"
server_key: "a0ffcc3b44befafc4af956d0aba0a338"
server_key_iv: "d8b32f81d4eeba74a43c256a3cf6a86b"
server_key_version: "encryptionkey@0"
I20260807 11:22:59.944293 18592 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 11:22:59.947912 18608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:22:59.949285 18592 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 11:22:59.949481 18592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "88ec7cf9488d40af9c742bd902fa595b"
format_stamp: "Formatted at 2026-08-07 11:22:59 on dist-test-slave-2w3w"
server_key: "a0ffcc3b44befafc4af956d0aba0a338"
server_key_iv: "d8b32f81d4eeba74a43c256a3cf6a86b"
server_key_version: "encryptionkey@0"
I20260807 11:22:59.949717 18592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:22:59.962788 18592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:22:59.967047 18592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:22:59.967339 18592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:22:59.970168 18592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:22:59.970288 18592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:22:59.970498 18592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:22:59.970587 18592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 11:22:59 dist-test-slave-2w3w krb5kdc[18318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101779, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:22:59.996887 18470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:59.970632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:44755 (local address 127.16.197.62:43575)
0807 11:22:59.970816 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 11:22:59.970823 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:22:59.972234 (+ 1411us) server_negotiation.cc:407] Connection header received
0807 11:22:59.974411 (+ 2177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:22:59.974419 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:22:59.974531 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:22:59.974665 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:22:59.977139 (+ 2474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:59.978498 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:59.981374 (+ 2876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:22:59.982226 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:22:59.982392 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:59.987988 (+ 5596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:22:59.987998 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:22:59.988004 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:22:59.988059 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:22:59.992840 (+ 4781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:59.994001 (+ 1161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:59.994005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:59.994008 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:59.994133 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:22:59.994629 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:22:59.994633 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:22:59.994635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:22:59.994941 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:22:59.995138 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 11:22:59.996267 (+ 1129us) server_negotiation.cc:299] Negotiation successful
0807 11:22:59.996506 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 11:22:59.998728 18617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:22:59.971311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:43575 (local address 127.16.197.2:44755)
0807 11:22:59.972061 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0807 11:22:59.972109 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 11:22:59.974011 (+ 1902us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:22:59.974942 (+ 931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:22:59.974988 (+ 46us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:22:59.975961 (+ 973us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:22:59.976856 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:22:59.976909 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:22:59.978716 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:22:59.978722 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:22:59.981201 (+ 2479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:22:59.981214 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:22:59.982609 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:22:59.982625 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:22:59.982941 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:22:59.983931 (+ 990us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:22:59.983973 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:22:59.987636 (+ 3663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:22:59.993051 (+ 5415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:22:59.993067 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:22:59.993091 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:22:59.993804 (+ 713us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:22:59.994259 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:22:59.994266 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:22:59.994269 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:22:59.994485 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:22:59.995112 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:22:59.995126 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:22:59.995741 (+ 615us) client_negotiation.cc:769] Sending connection context
0807 11:22:59.996120 (+ 379us) client_negotiation.cc:240] Negotiation successful
0807 11:22:59.996704 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":329,"threads_started":1}
I20260807 11:23:00.017673 18592 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:44257
I20260807 11:23:00.017810 18732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:44257 every 8 connection(s)
I20260807 11:23:00.019294 18592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:23:00.021829 18733 heartbeater.cc:347] Connected to a master server at 127.16.197.62:43575
I20260807 11:23:00.022081 18733 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:00.022943 18733 heartbeater.cc:511] Master 127.16.197.62:43575 requested a full tablet report, sending...
I20260807 11:23:00.024552 18367 ts_manager.cc:194] Registered new tserver with Master: 88ec7cf9488d40af9c742bd902fa595b (127.16.197.2:44257)
I20260807 11:23:00.025791 18367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.2@KRBTEST.COM'} at 127.16.197.2:44755
I20260807 11:23:00.028628 18470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:00.011917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:54891 (local address 127.16.197.62:43575)
0807 11:23:00.012155 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:00.012163 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:00.013132 (+ 969us) server_negotiation.cc:407] Connection header received
0807 11:23:00.013316 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:00.013325 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:00.013424 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:00.013561 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:00.015738 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.017028 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.018974 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.019629 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.019805 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:00.022078 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:00.022085 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:00.022090 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:00.022138 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:00.025705 (+ 3567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:00.026506 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:00.026512 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:00.026515 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:00.026682 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:00.027134 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:00.027140 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:00.027144 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:00.027481 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:00.027643 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:00.028055 (+ 412us) server_negotiation.cc:299] Negotiation successful
0807 11:23:00.028270 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 11:23:00.028770 18721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:00.012146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:43575 (local address 127.16.197.2:54891)
0807 11:23:00.012989 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:00.013016 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:00.013175 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:00.013620 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:00.013627 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:00.014450 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:00.015522 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:00.015549 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.017206 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.017241 (+ 35us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:00.018805 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:00.018819 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.019864 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.019872 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:00.020133 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:00.020919 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:00.020945 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:00.021863 (+ 918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:00.025843 (+ 3980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:00.025848 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:00.025850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:00.026384 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:00.026811 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:00.026819 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:00.026823 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:00.027016 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:00.027607 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:00.027612 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:00.027816 (+ 204us) client_negotiation.cc:769] Sending connection context
0807 11:23:00.027962 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 11:23:00.028284 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"thread_start_us":523,"threads_started":1}
I20260807 11:23:00.029479 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 18592
I20260807 11:23:00.029736 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance
I20260807 11:23:00.030247 17172 external_mini_cluster.cc:1428] Setting key 8ad5e6116e94d0d660d37cfa818a8912
WARNING: no policy specified for kudu/127.16.197.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
I20260807 11:23:00.075321 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:0
--local_ip_for_outbound_sockets=127.16.197.3
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:43575
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:44901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:00.404031 18737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:00.404333 18737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:00.404520 18737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:00.412125 18737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:00.412386 18737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:00.412597 18737 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 11:23:00.412760 18737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:23:00.424937 18737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:44901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.197.62:43575
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:00.426537 18737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:00.428187 18737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:00.443926 18743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:00.444952 18744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:00.445194 18746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:00.446110 18737 server_base.cc:1034] running on GCE node
I20260807 11:23:00.447425 18737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:00.448472 18737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:00.449791 18737 hybrid_clock.cc:648] HybridClock initialized: now 1786101780449666 us; error 33 us; skew 500 ppm
I20260807 11:23:00.450152 18737 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:00 dist-test-slave-2w3w krb5kdc[18318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101780, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:00.454401 18737 init.cc:377] Logged in from keytab as kudu/127.16.197.3@KRBTEST.COM (short username kudu)
I20260807 11:23:00.456403 18737 webserver.cc:460] Webserver started at http://127.16.197.3:42073/ using document root <none> and password file <none>
I20260807 11:23:00.457230 18737 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:00.457382 18737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:00.457767 18737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:00.461215 18737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-2/data/instance:
uuid: "0d57748dc8134476b31453604f247bc7"
format_stamp: "Formatted at 2026-08-07 11:23:00 on dist-test-slave-2w3w"
server_key: "c8b5932dc10d02e64e902cac1faf84f1"
server_key_iv: "04b7a70f562f621fc767865d18e4cf15"
server_key_version: "encryptionkey@0"
I20260807 11:23:00.462385 18737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance:
uuid: "0d57748dc8134476b31453604f247bc7"
format_stamp: "Formatted at 2026-08-07 11:23:00 on dist-test-slave-2w3w"
server_key: "c8b5932dc10d02e64e902cac1faf84f1"
server_key_iv: "04b7a70f562f621fc767865d18e4cf15"
server_key_version: "encryptionkey@0"
I20260807 11:23:00.468178 18737 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 11:23:00.471884 18753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:00.473215 18737 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 11:23:00.473409 18737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "0d57748dc8134476b31453604f247bc7"
format_stamp: "Formatted at 2026-08-07 11:23:00 on dist-test-slave-2w3w"
server_key: "c8b5932dc10d02e64e902cac1faf84f1"
server_key_iv: "04b7a70f562f621fc767865d18e4cf15"
server_key_version: "encryptionkey@0"
I20260807 11:23:00.473656 18737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:00.502008 18737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:00.506860 18737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:00.507189 18737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:00.510308 18737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:00.510421 18737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:00.510573 18737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:00.510654 18737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:00.521163 18588 heartbeater.cc:503] Master 127.16.197.62:43575 was elected leader, sending a full tablet report...
Aug 07 11:23:00 dist-test-slave-2w3w krb5kdc[18318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101780, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:00.536722 18470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:00.511014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:36437 (local address 127.16.197.62:43575)
0807 11:23:00.511241 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:00.511247 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:00.512687 (+ 1440us) server_negotiation.cc:407] Connection header received
0807 11:23:00.514826 (+ 2139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:00.514832 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:00.514996 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:00.515099 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:00.517881 (+ 2782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.519036 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.522347 (+ 3311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.523249 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.523423 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:00.528303 (+ 4880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:00.528323 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:00.528331 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:00.528384 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:00.532705 (+ 4321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:00.533514 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:00.533522 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:00.533528 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:00.533738 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:00.534177 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:00.534185 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:00.534191 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:00.534672 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:00.534895 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:00.535853 (+ 958us) server_negotiation.cc:299] Negotiation successful
0807 11:23:00.536177 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 11:23:00.537341 18762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:00.511715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:43575 (local address 127.16.197.3:36437)
0807 11:23:00.512516 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:00.512553 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:00.514492 (+ 1939us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:00.515417 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:00.515432 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:00.516350 (+ 918us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:00.517637 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:00.517675 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.519208 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.519216 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:00.522101 (+ 2885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:00.522122 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.523596 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.523602 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:00.523801 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:00.524644 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:00.524678 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:00.528022 (+ 3344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:00.532914 (+ 4892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:00.532927 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:00.532947 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:00.533391 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:00.533876 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:00.533879 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:00.533882 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:00.534066 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:00.534826 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:00.534832 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:00.535283 (+ 451us) client_negotiation.cc:769] Sending connection context
0807 11:23:00.535672 (+ 389us) client_negotiation.cc:240] Negotiation successful
0807 11:23:00.536100 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":370,"threads_started":1}
I20260807 11:23:00.565618 18737 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:40359
I20260807 11:23:00.565639 18876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:40359 every 8 connection(s)
I20260807 11:23:00.566920 18737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:23:00.569228 18877 heartbeater.cc:347] Connected to a master server at 127.16.197.62:43575
I20260807 11:23:00.569562 18877 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:00.570399 18877 heartbeater.cc:511] Master 127.16.197.62:43575 requested a full tablet report, sending...
I20260807 11:23:00.571628 18852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:00.554746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:43575 (local address 127.16.197.3:36317)
0807 11:23:00.555536 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:00.555554 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:00.555692 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:00.556087 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:00.556090 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:00.556780 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:00.557313 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:00.557338 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.559122 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.559129 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:00.561517 (+ 2388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:00.561539 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.562392 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.562401 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:00.562642 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:00.563581 (+ 939us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:00.563604 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:00.564909 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:00.568705 (+ 3796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:00.568709 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:00.568712 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:00.569235 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:00.569726 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:00.569729 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:00.569732 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:00.569895 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:00.570564 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:00.570575 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:00.570745 (+ 170us) client_negotiation.cc:769] Sending connection context
0807 11:23:00.570872 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 11:23:00.571192 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":481,"threads_started":1}
I20260807 11:23:00.571652 18470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:00.554549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:36317 (local address 127.16.197.62:43575)
0807 11:23:00.554799 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:00.554807 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:00.555671 (+ 864us) server_negotiation.cc:407] Connection header received
0807 11:23:00.555823 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:00.555831 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:00.555930 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:00.556057 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:00.557537 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.558873 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.561698 (+ 2825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.562143 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.562251 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:00.565178 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:00.565186 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:00.565190 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:00.565237 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:00.568549 (+ 3312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:00.569372 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:00.569378 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:00.569381 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:00.569554 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:00.570014 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:00.570019 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:00.570022 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:00.570418 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:00.570641 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:00.571033 (+ 392us) server_negotiation.cc:299] Negotiation successful
0807 11:23:00.571284 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 11:23:00.572209 18367 ts_manager.cc:194] Registered new tserver with Master: 0d57748dc8134476b31453604f247bc7 (127.16.197.3:40359)
I20260807 11:23:00.573194 18367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.3@KRBTEST.COM'} at 127.16.197.3:36437
I20260807 11:23:00.575826 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 18737
I20260807 11:23:00.576022 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance
I20260807 11:23:00.576470 17172 external_mini_cluster.cc:1428] Setting key e29fb907eb2728cc64ba06863585aedb
I20260807 11:23:00.579492 17172 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 11:23:00 dist-test-slave-2w3w krb5kdc[18318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101780, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 11:23:00 dist-test-slave-2w3w krb5kdc[18318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101780, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:00.630901 18470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:00.616054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46064 (local address 127.16.197.62:43575)
0807 11:23:00.616304 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:00.616312 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:00.616480 (+ 168us) server_negotiation.cc:407] Connection header received
0807 11:23:00.616636 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:00.616640 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:00.616708 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:00.616798 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:00.618383 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.619336 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.620735 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.621234 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.621332 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:00.625438 (+ 4106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:00.625443 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:00.625447 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:00.625483 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:00.628152 (+ 2669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:00.628880 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:00.628885 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:00.628889 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:00.629021 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:00.629379 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:00.629382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:00.629384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:00.629714 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:00.629914 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:00.630275 (+ 361us) server_negotiation.cc:299] Negotiation successful
0807 11:23:00.630466 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 11:23:00.635462 18367 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 11:23:00.642607 18367 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:23:00.673501 18892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:00.655243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:40359 (local address 127.0.0.1:47132)
0807 11:23:00.657835 (+ 2592us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:00.657859 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:00.658000 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:00.661346 (+ 3346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:00.661358 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:00.661400 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:00.661934 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:00.661951 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.666048 (+ 4097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.666057 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:00.670362 (+ 4305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:00.670375 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.672407 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.672413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:00.672624 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:00.672673 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 11:23:00.672825 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 11:23:00.673073 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2434,"thread_start_us":2191,"threads_started":1}
I20260807 11:23:00.674293 18894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:00.655693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47132 (local address 127.16.197.3:40359)
0807 11:23:00.658770 (+ 3077us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:00.658783 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:00.658822 (+ 39us) server_negotiation.cc:407] Connection header received
0807 11:23:00.658943 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:00.658953 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:00.660634 (+ 1681us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:00.660868 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:00.662232 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.665825 (+ 3593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.670536 (+ 4711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.672199 (+ 1663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.672334 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:00.672583 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:00.673760 (+ 1177us) server_negotiation.cc:299] Negotiation successful
0807 11:23:00.673954 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2845,"thread_start_us":2641,"threads_started":1}
I20260807 11:23:00.678170 18896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:00.658004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41420 (local address 127.16.197.1:35649)
0807 11:23:00.669063 (+ 11059us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:00.669076 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:00.670080 (+ 1004us) server_negotiation.cc:407] Connection header received
0807 11:23:00.670196 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:00.670204 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:00.670491 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:00.670718 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:00.671582 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.672776 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.675850 (+ 3074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.676974 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.677125 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:00.677395 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:00.677605 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 11:23:00.677769 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10827,"thread_start_us":11065,"threads_started":1}
I20260807 11:23:00.678208 18891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:00.655078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:44257 (local address 127.0.0.1:40508)
0807 11:23:00.656020 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:00.656081 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:00.656277 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:00.670749 (+ 14472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:00.670753 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:00.670780 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:00.671469 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:00.671481 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.673614 (+ 2133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.673621 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:00.675314 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:00.675327 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.677352 (+ 2025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.677360 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:00.677531 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:00.677561 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 11:23:00.677641 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 11:23:00.677784 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"mutex_wait_us":76,"thread_start_us":494,"threads_started":1}
I20260807 11:23:00.678226 18895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:00.657328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:35649 (local address 127.0.0.1:41420)
0807 11:23:00.669770 (+ 12442us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:00.669794 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:00.669933 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:00.670707 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:00.670714 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:00.670741 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:00.671398 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:00.671412 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.672974 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.672979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:00.675697 (+ 2718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:00.675710 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.677178 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.677185 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:00.677398 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:00.677441 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 11:23:00.677549 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 11:23:00.677689 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12298,"mutex_wait_us":34,"thread_start_us":12150,"threads_started":1}
I20260807 11:23:00.679105 18893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:00.655249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40508 (local address 127.16.197.2:44257)
0807 11:23:00.670198 (+ 14949us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:00.670206 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:00.670231 (+ 25us) server_negotiation.cc:407] Connection header received
0807 11:23:00.670313 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:00.670319 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:00.670491 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:00.671080 (+ 589us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:00.671687 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.673436 (+ 1749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.675465 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.676830 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.676976 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:00.677175 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:00.678587 (+ 1412us) server_negotiation.cc:299] Negotiation successful
0807 11:23:00.678768 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14790,"thread_start_us":14647,"threads_started":1}
I20260807 11:23:00.678866 18805 tablet_service.cc:1467] Processing CreateTablet for tablet 727481ca01604e5ca7eae410b126baa3 (DEFAULT_TABLE table=test-table [id=6d771e09a6094a7f8d36fa44dcceb7af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:00.680991 18805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 727481ca01604e5ca7eae410b126baa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:00.681813 18513 tablet_service.cc:1467] Processing CreateTablet for tablet 727481ca01604e5ca7eae410b126baa3 (DEFAULT_TABLE table=test-table [id=6d771e09a6094a7f8d36fa44dcceb7af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:00.682855 18513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 727481ca01604e5ca7eae410b126baa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:00.682423 18659 tablet_service.cc:1467] Processing CreateTablet for tablet 727481ca01604e5ca7eae410b126baa3 (DEFAULT_TABLE table=test-table [id=6d771e09a6094a7f8d36fa44dcceb7af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:00.683734 18659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 727481ca01604e5ca7eae410b126baa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:00.694602 18897 tablet_bootstrap.cc:492] T 727481ca01604e5ca7eae410b126baa3 P 257609186abd4d8eb658659a01aa71ab: Bootstrap starting.
I20260807 11:23:00.696800 18898 tablet_bootstrap.cc:492] T 727481ca01604e5ca7eae410b126baa3 P 0d57748dc8134476b31453604f247bc7: Bootstrap starting.
I20260807 11:23:00.698355 18899 tablet_bootstrap.cc:492] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b: Bootstrap starting.
I20260807 11:23:00.701814 18898 tablet_bootstrap.cc:654] T 727481ca01604e5ca7eae410b126baa3 P 0d57748dc8134476b31453604f247bc7: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:00.702414 18899 tablet_bootstrap.cc:654] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:00.703874 18897 tablet_bootstrap.cc:654] T 727481ca01604e5ca7eae410b126baa3 P 257609186abd4d8eb658659a01aa71ab: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:00.703963 18898 log.cc:824] T 727481ca01604e5ca7eae410b126baa3 P 0d57748dc8134476b31453604f247bc7: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:00.704345 18899 log.cc:824] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:00.706449 18897 log.cc:824] T 727481ca01604e5ca7eae410b126baa3 P 257609186abd4d8eb658659a01aa71ab: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:00.714555 18898 tablet_bootstrap.cc:492] T 727481ca01604e5ca7eae410b126baa3 P 0d57748dc8134476b31453604f247bc7: No bootstrap required, opened a new log
I20260807 11:23:00.715090 18898 ts_tablet_manager.cc:1397] T 727481ca01604e5ca7eae410b126baa3 P 0d57748dc8134476b31453604f247bc7: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.009s
I20260807 11:23:00.715999 18899 tablet_bootstrap.cc:492] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b: No bootstrap required, opened a new log
I20260807 11:23:00.716411 18899 ts_tablet_manager.cc:1397] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b: Time spent bootstrapping tablet: real 0.018s user 0.011s sys 0.003s
I20260807 11:23:00.723491 18897 tablet_bootstrap.cc:492] T 727481ca01604e5ca7eae410b126baa3 P 257609186abd4d8eb658659a01aa71ab: No bootstrap required, opened a new log
I20260807 11:23:00.723944 18897 ts_tablet_manager.cc:1397] T 727481ca01604e5ca7eae410b126baa3 P 257609186abd4d8eb658659a01aa71ab: Time spent bootstrapping tablet: real 0.029s user 0.014s sys 0.008s
I20260807 11:23:00.731468 18898 raft_consensus.cc:348] T 727481ca01604e5ca7eae410b126baa3 P 0d57748dc8134476b31453604f247bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } } peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } }
I20260807 11:23:00.732138 18898 raft_consensus.cc:374] T 727481ca01604e5ca7eae410b126baa3 P 0d57748dc8134476b31453604f247bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:00.732299 18898 raft_consensus.cc:729] T 727481ca01604e5ca7eae410b126baa3 P 0d57748dc8134476b31453604f247bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d57748dc8134476b31453604f247bc7, State: Initialized, Role: FOLLOWER
I20260807 11:23:00.733134 18898 consensus_queue.cc:260] T 727481ca01604e5ca7eae410b126baa3 P 0d57748dc8134476b31453604f247bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } } peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } }
I20260807 11:23:00.733253 18899 raft_consensus.cc:348] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } } peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } }
I20260807 11:23:00.733783 18899 raft_consensus.cc:374] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:00.733938 18899 raft_consensus.cc:729] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ec7cf9488d40af9c742bd902fa595b, State: Initialized, Role: FOLLOWER
I20260807 11:23:00.734860 18877 heartbeater.cc:503] Master 127.16.197.62:43575 was elected leader, sending a full tablet report...
I20260807 11:23:00.734755 18899 consensus_queue.cc:260] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } } peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } }
I20260807 11:23:00.735739 18898 ts_tablet_manager.cc:1428] T 727481ca01604e5ca7eae410b126baa3 P 0d57748dc8134476b31453604f247bc7: Time spent starting tablet: real 0.020s user 0.012s sys 0.005s
I20260807 11:23:00.737054 18733 heartbeater.cc:503] Master 127.16.197.62:43575 was elected leader, sending a full tablet report...
I20260807 11:23:00.737807 18899 ts_tablet_manager.cc:1428] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b: Time spent starting tablet: real 0.021s user 0.008s sys 0.012s
I20260807 11:23:00.740031 18897 raft_consensus.cc:348] T 727481ca01604e5ca7eae410b126baa3 P 257609186abd4d8eb658659a01aa71ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } } peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } }
I20260807 11:23:00.740583 18897 raft_consensus.cc:374] T 727481ca01604e5ca7eae410b126baa3 P 257609186abd4d8eb658659a01aa71ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:00.740742 18897 raft_consensus.cc:729] T 727481ca01604e5ca7eae410b126baa3 P 257609186abd4d8eb658659a01aa71ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 257609186abd4d8eb658659a01aa71ab, State: Initialized, Role: FOLLOWER
I20260807 11:23:00.741505 18897 consensus_queue.cc:260] T 727481ca01604e5ca7eae410b126baa3 P 257609186abd4d8eb658659a01aa71ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } } peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } }
I20260807 11:23:00.743862 18897 ts_tablet_manager.cc:1428] T 727481ca01604e5ca7eae410b126baa3 P 257609186abd4d8eb658659a01aa71ab: Time spent starting tablet: real 0.020s user 0.015s sys 0.004s
W20260807 11:23:00.754565 18589 tablet.cc:2369] T 727481ca01604e5ca7eae410b126baa3 P 257609186abd4d8eb658659a01aa71ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 11:23:00.772181 18734 tablet.cc:2369] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:23:00.789258 18905 raft_consensus.cc:482] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:23:00.789749 18905 raft_consensus.cc:504] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } } peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } }
I20260807 11:23:00.791903 18905 leader_election.cc:288] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d57748dc8134476b31453604f247bc7 (127.16.197.3:40359), 257609186abd4d8eb658659a01aa71ab (127.16.197.1:35649)
I20260807 11:23:00.802033 18894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:00.792394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:53515 (local address 127.16.197.3:40359)
0807 11:23:00.792792 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:00.792801 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:00.793652 (+ 851us) server_negotiation.cc:407] Connection header received
0807 11:23:00.793904 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:00.793908 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:00.793968 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:00.794065 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:00.795037 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.796374 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.799272 (+ 2898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.800773 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.801023 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:00.801083 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:00.801520 (+ 437us) server_negotiation.cc:299] Negotiation successful
0807 11:23:00.801619 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260807 11:23:00.802438 18907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:00.792059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:40359 (local address 127.16.197.2:53515)
0807 11:23:00.793532 (+ 1473us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:00.793548 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:00.793737 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:00.794086 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:00.794094 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:00.794121 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:00.794828 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:00.794845 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.796504 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.796509 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:00.799081 (+ 2572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:00.799094 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.801089 (+ 1995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.801096 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:00.801312 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:00.801350 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 11:23:00.801492 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 11:23:00.801652 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1328,"mutex_wait_us":1,"thread_start_us":787,"threads_started":1}
I20260807 11:23:00.803232 18908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:00.793014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:35649 (local address 127.16.197.2:56061)
0807 11:23:00.794596 (+ 1582us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:00.794611 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:00.794742 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:00.795168 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:00.795174 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:00.795201 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:00.795975 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:00.795990 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.797500 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.797504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:00.800155 (+ 2651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:00.800172 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.802063 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.802074 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:00.802284 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:00.802341 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 11:23:00.802458 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 11:23:00.802624 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1380,"thread_start_us":1020,"threads_started":1}
I20260807 11:23:00.803574 18896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:00.792898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:56061 (local address 127.16.197.1:35649)
0807 11:23:00.793103 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:00.793107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:00.794727 (+ 1620us) server_negotiation.cc:407] Connection header received
0807 11:23:00.794903 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:00.794911 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:00.795049 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:00.795120 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:00.796145 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.797313 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.800359 (+ 3046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.801602 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.801766 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:00.801859 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:00.802487 (+ 628us) server_negotiation.cc:299] Negotiation successful
0807 11:23:00.802624 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 11:23:00.804128 18829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "727481ca01604e5ca7eae410b126baa3" candidate_uuid: "88ec7cf9488d40af9c742bd902fa595b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d57748dc8134476b31453604f247bc7" is_pre_election: true
I20260807 11:23:00.804991 18829 raft_consensus.cc:2454] T 727481ca01604e5ca7eae410b126baa3 P 0d57748dc8134476b31453604f247bc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ec7cf9488d40af9c742bd902fa595b in term 0.
I20260807 11:23:00.805056 18535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "727481ca01604e5ca7eae410b126baa3" candidate_uuid: "88ec7cf9488d40af9c742bd902fa595b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "257609186abd4d8eb658659a01aa71ab" is_pre_election: true
I20260807 11:23:00.805658 18535 raft_consensus.cc:2454] T 727481ca01604e5ca7eae410b126baa3 P 257609186abd4d8eb658659a01aa71ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ec7cf9488d40af9c742bd902fa595b in term 0.
I20260807 11:23:00.806054 18610 leader_election.cc:302] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d57748dc8134476b31453604f247bc7, 88ec7cf9488d40af9c742bd902fa595b; no voters:
I20260807 11:23:00.806598 18905 raft_consensus.cc:2781] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:23:00.806795 18905 raft_consensus.cc:482] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:23:00.806941 18905 raft_consensus.cc:3037] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:00.809777 18905 raft_consensus.cc:504] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } } peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } }
I20260807 11:23:00.810889 18905 leader_election.cc:288] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b [CANDIDATE]: Term 1 election: Requested vote from peers 0d57748dc8134476b31453604f247bc7 (127.16.197.3:40359), 257609186abd4d8eb658659a01aa71ab (127.16.197.1:35649)
I20260807 11:23:00.811604 18535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "727481ca01604e5ca7eae410b126baa3" candidate_uuid: "88ec7cf9488d40af9c742bd902fa595b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "257609186abd4d8eb658659a01aa71ab"
I20260807 11:23:00.811676 18829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "727481ca01604e5ca7eae410b126baa3" candidate_uuid: "88ec7cf9488d40af9c742bd902fa595b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d57748dc8134476b31453604f247bc7"
I20260807 11:23:00.811861 18535 raft_consensus.cc:3037] T 727481ca01604e5ca7eae410b126baa3 P 257609186abd4d8eb658659a01aa71ab [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:00.812034 18829 raft_consensus.cc:3037] T 727481ca01604e5ca7eae410b126baa3 P 0d57748dc8134476b31453604f247bc7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:00.816226 18535 raft_consensus.cc:2454] T 727481ca01604e5ca7eae410b126baa3 P 257609186abd4d8eb658659a01aa71ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ec7cf9488d40af9c742bd902fa595b in term 1.
I20260807 11:23:00.816830 18609 leader_election.cc:302] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 257609186abd4d8eb658659a01aa71ab, 88ec7cf9488d40af9c742bd902fa595b; no voters:
I20260807 11:23:00.817377 18905 raft_consensus.cc:2781] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:00.817451 18829 raft_consensus.cc:2454] T 727481ca01604e5ca7eae410b126baa3 P 0d57748dc8134476b31453604f247bc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ec7cf9488d40af9c742bd902fa595b in term 1.
I20260807 11:23:00.818250 18905 raft_consensus.cc:686] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b [term 1 LEADER]: Becoming Leader. State: Replica: 88ec7cf9488d40af9c742bd902fa595b, State: Running, Role: LEADER
W20260807 11:23:00.818926 18878 tablet.cc:2369] T 727481ca01604e5ca7eae410b126baa3 P 0d57748dc8134476b31453604f247bc7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:23:00.818856 18905 consensus_queue.cc:237] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } } peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } }
I20260807 11:23:00.825007 18366 catalog_manager.cc:5495] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b reported cstate change: term changed from 0 to 1, leader changed from <none> to 88ec7cf9488d40af9c742bd902fa595b (127.16.197.2). New cstate: current_term: 1 leader_uuid: "88ec7cf9488d40af9c742bd902fa595b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } health_report { overall_health: UNKNOWN } } }
I20260807 11:23:00.874460 18893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:00.867830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40520 (local address 127.16.197.2:44257)
0807 11:23:00.868085 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:00.868091 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:00.868267 (+ 176us) server_negotiation.cc:407] Connection header received
0807 11:23:00.868566 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:00.868570 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:00.868673 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:00.868810 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:23:00.869229 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.870093 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.871749 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:00.872350 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:00.872459 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:00.873170 (+ 711us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:23:00.873695 (+ 525us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:23:00.873838 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:00.874010 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 11:23:00.874112 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 11:23:00.883927 18829 raft_consensus.cc:1261] T 727481ca01604e5ca7eae410b126baa3 P 0d57748dc8134476b31453604f247bc7 [term 1 FOLLOWER]: Refusing update from remote peer 88ec7cf9488d40af9c742bd902fa595b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:00.884203 18535 raft_consensus.cc:1261] T 727481ca01604e5ca7eae410b126baa3 P 257609186abd4d8eb658659a01aa71ab [term 1 FOLLOWER]: Refusing update from remote peer 88ec7cf9488d40af9c742bd902fa595b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:00.885118 18909 consensus_queue.cc:1035] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 11:23:00.885802 18905 consensus_queue.cc:1035] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b [LEADER]: Connected to new peer: Peer: permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:00.907197 18912 mvcc.cc:205] Tried to move back new op lower bound from 7315872894490128384 to 7315872894239309824. Current Snapshot: MvccSnapshot[applied={T|T < 7315872894490128384}]
I20260807 11:23:00.931980 18366 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:23:00.932446 18366 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:23:00.937193 18366 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b527b68d53bf4086bc4f8b2f7627b6c5: Sending DeleteTablet for 3 replicas of tablet 727481ca01604e5ca7eae410b126baa3
I20260807 11:23:00.938875 18659 tablet_service.cc:1514] Processing DeleteTablet for tablet 727481ca01604e5ca7eae410b126baa3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:00 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:40508
I20260807 11:23:00.938911 18805 tablet_service.cc:1514] Processing DeleteTablet for tablet 727481ca01604e5ca7eae410b126baa3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:00 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:47132
I20260807 11:23:00.940716 18513 tablet_service.cc:1514] Processing DeleteTablet for tablet 727481ca01604e5ca7eae410b126baa3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:00 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:41420
I20260807 11:23:00.942116 18365 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 11:23:00.943907 18365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:23:00.951155 18923 tablet_replica.cc:330] T 727481ca01604e5ca7eae410b126baa3 P 0d57748dc8134476b31453604f247bc7: stopping tablet replica
I20260807 11:23:00.951817 18924 tablet_replica.cc:330] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b: stopping tablet replica
I20260807 11:23:00.951905 18923 raft_consensus.cc:2229] T 727481ca01604e5ca7eae410b126baa3 P 0d57748dc8134476b31453604f247bc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:00.952284 18924 raft_consensus.cc:2229] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b [term 1 LEADER]: Raft consensus shutting down.
I20260807 11:23:00.952661 18923 raft_consensus.cc:2258] T 727481ca01604e5ca7eae410b126baa3 P 0d57748dc8134476b31453604f247bc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:00.953320 18924 raft_consensus.cc:2258] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:00.954692 18804 tablet_service.cc:1467] Processing CreateTablet for tablet 9ade140daff04ea384c4dc99cd26b70d (DEFAULT_TABLE table=test-table [id=3cafb041540a42e591de6240a4bad8d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:00.955519 18804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ade140daff04ea384c4dc99cd26b70d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:00.955757 18924 ts_tablet_manager.cc:1905] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:00.958400 18926 tablet_replica.cc:330] T 727481ca01604e5ca7eae410b126baa3 P 257609186abd4d8eb658659a01aa71ab: stopping tablet replica
I20260807 11:23:00.958921 18926 raft_consensus.cc:2229] T 727481ca01604e5ca7eae410b126baa3 P 257609186abd4d8eb658659a01aa71ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:00.960107 18926 raft_consensus.cc:2258] T 727481ca01604e5ca7eae410b126baa3 P 257609186abd4d8eb658659a01aa71ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:00.960338 18512 tablet_service.cc:1467] Processing CreateTablet for tablet 9ade140daff04ea384c4dc99cd26b70d (DEFAULT_TABLE table=test-table [id=3cafb041540a42e591de6240a4bad8d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:00.961151 18512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ade140daff04ea384c4dc99cd26b70d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:00.962478 18926 ts_tablet_manager.cc:1905] T 727481ca01604e5ca7eae410b126baa3 P 257609186abd4d8eb658659a01aa71ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:00.969992 18897 tablet_bootstrap.cc:492] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab: Bootstrap starting.
I20260807 11:23:00.970304 18658 tablet_service.cc:1467] Processing CreateTablet for tablet 9ade140daff04ea384c4dc99cd26b70d (DEFAULT_TABLE table=test-table [id=3cafb041540a42e591de6240a4bad8d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:00.971204 18658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ade140daff04ea384c4dc99cd26b70d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:00.972509 18897 tablet_bootstrap.cc:654] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:00.973706 18898 tablet_bootstrap.cc:492] T 9ade140daff04ea384c4dc99cd26b70d P 0d57748dc8134476b31453604f247bc7: Bootstrap starting.
I20260807 11:23:00.976321 18898 tablet_bootstrap.cc:654] T 9ade140daff04ea384c4dc99cd26b70d P 0d57748dc8134476b31453604f247bc7: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:00.977051 18926 ts_tablet_manager.cc:1918] T 727481ca01604e5ca7eae410b126baa3 P 257609186abd4d8eb658659a01aa71ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:23:00.977344 18926 log.cc:1196] T 727481ca01604e5ca7eae410b126baa3 P 257609186abd4d8eb658659a01aa71ab: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-0/wal/wals/727481ca01604e5ca7eae410b126baa3
I20260807 11:23:00.978134 18926 ts_tablet_manager.cc:1939] T 727481ca01604e5ca7eae410b126baa3 P 257609186abd4d8eb658659a01aa71ab: Deleting consensus metadata
I20260807 11:23:00.981030 18899 tablet_bootstrap.cc:492] T 9ade140daff04ea384c4dc99cd26b70d P 88ec7cf9488d40af9c742bd902fa595b: Bootstrap starting.
I20260807 11:23:00.984368 18899 tablet_bootstrap.cc:654] T 9ade140daff04ea384c4dc99cd26b70d P 88ec7cf9488d40af9c742bd902fa595b: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:00.985982 18898 tablet_bootstrap.cc:492] T 9ade140daff04ea384c4dc99cd26b70d P 0d57748dc8134476b31453604f247bc7: No bootstrap required, opened a new log
I20260807 11:23:00.986294 18898 ts_tablet_manager.cc:1397] T 9ade140daff04ea384c4dc99cd26b70d P 0d57748dc8134476b31453604f247bc7: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.007s
I20260807 11:23:00.986567 18350 catalog_manager.cc:4841] TS 257609186abd4d8eb658659a01aa71ab (127.16.197.1:35649): tablet 727481ca01604e5ca7eae410b126baa3 (table test-table [id=6d771e09a6094a7f8d36fa44dcceb7af]) successfully deleted
I20260807 11:23:00.987903 18898 raft_consensus.cc:348] T 9ade140daff04ea384c4dc99cd26b70d P 0d57748dc8134476b31453604f247bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } }
I20260807 11:23:00.988715 18898 raft_consensus.cc:374] T 9ade140daff04ea384c4dc99cd26b70d P 0d57748dc8134476b31453604f247bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:00.988929 18898 raft_consensus.cc:729] T 9ade140daff04ea384c4dc99cd26b70d P 0d57748dc8134476b31453604f247bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d57748dc8134476b31453604f247bc7, State: Initialized, Role: FOLLOWER
I20260807 11:23:00.989164 18897 tablet_bootstrap.cc:492] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab: No bootstrap required, opened a new log
I20260807 11:23:00.989378 18897 ts_tablet_manager.cc:1397] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab: Time spent bootstrapping tablet: real 0.020s user 0.009s sys 0.004s
I20260807 11:23:00.989430 18898 consensus_queue.cc:260] T 9ade140daff04ea384c4dc99cd26b70d P 0d57748dc8134476b31453604f247bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } }
I20260807 11:23:00.990484 18897 raft_consensus.cc:348] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } }
I20260807 11:23:00.990669 18898 ts_tablet_manager.cc:1428] T 9ade140daff04ea384c4dc99cd26b70d P 0d57748dc8134476b31453604f247bc7: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260807 11:23:00.990792 18897 raft_consensus.cc:374] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:00.990998 18897 raft_consensus.cc:729] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 257609186abd4d8eb658659a01aa71ab, State: Initialized, Role: FOLLOWER
I20260807 11:23:00.991429 18923 ts_tablet_manager.cc:1905] T 727481ca01604e5ca7eae410b126baa3 P 0d57748dc8134476b31453604f247bc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:00.991464 18897 consensus_queue.cc:260] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } }
I20260807 11:23:00.992187 18897 ts_tablet_manager.cc:1428] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 11:23:00.997821 18924 ts_tablet_manager.cc:1918] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:23:00.998111 18924 log.cc:1196] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-1/wal/wals/727481ca01604e5ca7eae410b126baa3
I20260807 11:23:00.999020 18924 ts_tablet_manager.cc:1939] T 727481ca01604e5ca7eae410b126baa3 P 88ec7cf9488d40af9c742bd902fa595b: Deleting consensus metadata
I20260807 11:23:01.001922 18352 catalog_manager.cc:4841] TS 88ec7cf9488d40af9c742bd902fa595b (127.16.197.2:44257): tablet 727481ca01604e5ca7eae410b126baa3 (table test-table [id=6d771e09a6094a7f8d36fa44dcceb7af]) successfully deleted
I20260807 11:23:01.006532 18923 ts_tablet_manager.cc:1918] T 727481ca01604e5ca7eae410b126baa3 P 0d57748dc8134476b31453604f247bc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:23:01.006834 18923 log.cc:1196] T 727481ca01604e5ca7eae410b126baa3 P 0d57748dc8134476b31453604f247bc7: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-2/wal/wals/727481ca01604e5ca7eae410b126baa3
I20260807 11:23:01.007730 18923 ts_tablet_manager.cc:1939] T 727481ca01604e5ca7eae410b126baa3 P 0d57748dc8134476b31453604f247bc7: Deleting consensus metadata
I20260807 11:23:01.008585 18899 tablet_bootstrap.cc:492] T 9ade140daff04ea384c4dc99cd26b70d P 88ec7cf9488d40af9c742bd902fa595b: No bootstrap required, opened a new log
I20260807 11:23:01.008839 18899 ts_tablet_manager.cc:1397] T 9ade140daff04ea384c4dc99cd26b70d P 88ec7cf9488d40af9c742bd902fa595b: Time spent bootstrapping tablet: real 0.028s user 0.000s sys 0.012s
I20260807 11:23:01.010203 18351 catalog_manager.cc:4841] TS 0d57748dc8134476b31453604f247bc7 (127.16.197.3:40359): tablet 727481ca01604e5ca7eae410b126baa3 (table test-table [id=6d771e09a6094a7f8d36fa44dcceb7af]) successfully deleted
I20260807 11:23:01.010241 18899 raft_consensus.cc:348] T 9ade140daff04ea384c4dc99cd26b70d P 88ec7cf9488d40af9c742bd902fa595b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } }
I20260807 11:23:01.010600 18899 raft_consensus.cc:374] T 9ade140daff04ea384c4dc99cd26b70d P 88ec7cf9488d40af9c742bd902fa595b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:01.010828 18899 raft_consensus.cc:729] T 9ade140daff04ea384c4dc99cd26b70d P 88ec7cf9488d40af9c742bd902fa595b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ec7cf9488d40af9c742bd902fa595b, State: Initialized, Role: FOLLOWER
I20260807 11:23:01.011248 18899 consensus_queue.cc:260] T 9ade140daff04ea384c4dc99cd26b70d P 88ec7cf9488d40af9c742bd902fa595b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } }
I20260807 11:23:01.012014 18899 ts_tablet_manager.cc:1428] T 9ade140daff04ea384c4dc99cd26b70d P 88ec7cf9488d40af9c742bd902fa595b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 11:23:01.021003 18906 raft_consensus.cc:482] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:23:01.021337 18906 raft_consensus.cc:504] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } }
I20260807 11:23:01.024053 18906 leader_election.cc:288] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ec7cf9488d40af9c742bd902fa595b (127.16.197.2:44257), 0d57748dc8134476b31453604f247bc7 (127.16.197.3:40359)
I20260807 11:23:01.050329 18893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:01.024525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:49295 (local address 127.16.197.2:44257)
0807 11:23:01.024693 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:01.024699 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:01.036458 (+ 11759us) server_negotiation.cc:407] Connection header received
0807 11:23:01.038588 (+ 2130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:01.038593 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:01.038672 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:01.038753 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:01.040049 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.041637 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.047070 (+ 5433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.048722 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.048922 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:01.049005 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:01.049786 (+ 781us) server_negotiation.cc:299] Negotiation successful
0807 11:23:01.049927 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 11:23:01.051635 18931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:01.024217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:44257 (local address 127.16.197.1:49295)
0807 11:23:01.036131 (+ 11914us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:01.036161 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:01.038387 (+ 2226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:01.039021 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:01.039026 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:01.039053 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:01.039733 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:01.039749 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.041898 (+ 2149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.041917 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:01.046790 (+ 4873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:01.046808 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.049314 (+ 2506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.049322 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:01.049559 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:01.049597 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 11:23:01.050761 (+ 1164us) client_negotiation.cc:240] Negotiation successful
0807 11:23:01.050928 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11705,"mutex_wait_us":2254,"thread_start_us":11371,"threads_started":1}
I20260807 11:23:01.054397 18894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:01.025115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:43797 (local address 127.16.197.3:40359)
0807 11:23:01.025301 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:01.025307 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:01.036865 (+ 11558us) server_negotiation.cc:407] Connection header received
0807 11:23:01.037368 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:01.037375 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:01.037518 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:01.037590 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:01.038981 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.040541 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.051146 (+ 10605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.052510 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.052708 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:01.052813 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:01.053872 (+ 1059us) server_negotiation.cc:299] Negotiation successful
0807 11:23:01.054040 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 11:23:01.054685 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:01.024702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:40359 (local address 127.16.197.1:43797)
0807 11:23:01.036741 (+ 12039us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:01.036762 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:01.037022 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:01.037854 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:01.037860 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:01.037889 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:01.038730 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:01.038744 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.040763 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.040769 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:01.044636 (+ 3867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:01.044655 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.053222 (+ 8567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.053231 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:01.053540 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:01.053643 (+ 103us) client_negotiation.cc:769] Sending connection context
0807 11:23:01.053773 (+ 130us) client_negotiation.cc:240] Negotiation successful
0807 11:23:01.054002 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11844,"thread_start_us":11496,"threads_started":1}
I20260807 11:23:01.055922 18829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ade140daff04ea384c4dc99cd26b70d" candidate_uuid: "257609186abd4d8eb658659a01aa71ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d57748dc8134476b31453604f247bc7" is_pre_election: true
I20260807 11:23:01.055902 18680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ade140daff04ea384c4dc99cd26b70d" candidate_uuid: "257609186abd4d8eb658659a01aa71ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ec7cf9488d40af9c742bd902fa595b" is_pre_election: true
I20260807 11:23:01.056298 18829 raft_consensus.cc:2454] T 9ade140daff04ea384c4dc99cd26b70d P 0d57748dc8134476b31453604f247bc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 257609186abd4d8eb658659a01aa71ab in term 0.
I20260807 11:23:01.056514 18680 raft_consensus.cc:2454] T 9ade140daff04ea384c4dc99cd26b70d P 88ec7cf9488d40af9c742bd902fa595b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 257609186abd4d8eb658659a01aa71ab in term 0.
I20260807 11:23:01.057631 18463 leader_election.cc:302] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0d57748dc8134476b31453604f247bc7, 257609186abd4d8eb658659a01aa71ab, 88ec7cf9488d40af9c742bd902fa595b; no voters:
I20260807 11:23:01.058287 18906 raft_consensus.cc:2781] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:23:01.058494 18906 raft_consensus.cc:482] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:23:01.058648 18906 raft_consensus.cc:3037] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:01.061892 18906 raft_consensus.cc:504] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } }
I20260807 11:23:01.063009 18906 leader_election.cc:288] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab [CANDIDATE]: Term 1 election: Requested vote from peers 88ec7cf9488d40af9c742bd902fa595b (127.16.197.2:44257), 0d57748dc8134476b31453604f247bc7 (127.16.197.3:40359)
I20260807 11:23:01.064041 18680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ade140daff04ea384c4dc99cd26b70d" candidate_uuid: "257609186abd4d8eb658659a01aa71ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ec7cf9488d40af9c742bd902fa595b"
I20260807 11:23:01.064330 18680 raft_consensus.cc:3037] T 9ade140daff04ea384c4dc99cd26b70d P 88ec7cf9488d40af9c742bd902fa595b [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:01.065068 18829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ade140daff04ea384c4dc99cd26b70d" candidate_uuid: "257609186abd4d8eb658659a01aa71ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d57748dc8134476b31453604f247bc7"
I20260807 11:23:01.065438 18829 raft_consensus.cc:3037] T 9ade140daff04ea384c4dc99cd26b70d P 0d57748dc8134476b31453604f247bc7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:01.069836 18829 raft_consensus.cc:2454] T 9ade140daff04ea384c4dc99cd26b70d P 0d57748dc8134476b31453604f247bc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 257609186abd4d8eb658659a01aa71ab in term 1.
I20260807 11:23:01.070477 18680 raft_consensus.cc:2454] T 9ade140daff04ea384c4dc99cd26b70d P 88ec7cf9488d40af9c742bd902fa595b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 257609186abd4d8eb658659a01aa71ab in term 1.
I20260807 11:23:01.070782 18463 leader_election.cc:302] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d57748dc8134476b31453604f247bc7, 257609186abd4d8eb658659a01aa71ab; no voters:
I20260807 11:23:01.071449 18906 raft_consensus.cc:2781] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:01.075579 18906 raft_consensus.cc:686] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab [term 1 LEADER]: Becoming Leader. State: Replica: 257609186abd4d8eb658659a01aa71ab, State: Running, Role: LEADER
I20260807 11:23:01.076357 18906 consensus_queue.cc:237] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } }
I20260807 11:23:01.081485 18366 catalog_manager.cc:5495] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 257609186abd4d8eb658659a01aa71ab (127.16.197.1). New cstate: current_term: 1 leader_uuid: "257609186abd4d8eb658659a01aa71ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } health_report { overall_health: UNKNOWN } } }
Aug 07 11:23:01 dist-test-slave-2w3w krb5kdc[18318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101778, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:01.149821 18940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:01.126989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:43575 (local address 127.0.0.1:46066)
0807 11:23:01.128328 (+ 1339us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:01.128353 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:01.128621 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:01.129112 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:01.129117 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:01.130568 (+ 1451us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:01.131182 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:01.131199 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.133441 (+ 2242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.133449 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:01.135882 (+ 2433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:01.135899 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.136878 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.136886 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:01.137207 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:01.138232 (+ 1025us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:01.138263 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:01.142191 (+ 3928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:01.146396 (+ 4205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:01.146410 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:01.146418 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:01.146910 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:01.147447 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:01.147454 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:01.147457 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:01.147721 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:01.148365 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:01.148396 (+ 31us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:01.148616 (+ 220us) client_negotiation.cc:769] Sending connection context
0807 11:23:01.148840 (+ 224us) client_negotiation.cc:240] Negotiation successful
0807 11:23:01.149197 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1162,"thread_start_us":759,"threads_started":1}
I20260807 11:23:01.149927 18470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:01.126788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46066 (local address 127.16.197.62:43575)
0807 11:23:01.127032 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:01.127041 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:01.128513 (+ 1472us) server_negotiation.cc:407] Connection header received
0807 11:23:01.128777 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:01.128785 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:01.128884 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:01.128994 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:01.131392 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.133259 (+ 1867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.136059 (+ 2800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.136605 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.136708 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:01.142469 (+ 5761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:01.142476 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:01.142479 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:01.142512 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:01.146221 (+ 3709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:01.147112 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:01.147120 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:01.147126 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:01.147327 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:01.147851 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:01.147856 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:01.147859 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:01.148222 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:01.148449 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:01.148975 (+ 526us) server_negotiation.cc:299] Negotiation successful
0807 11:23:01.149395 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":146,"server-negotiator.queue_time_us":78}
I20260807 11:23:01.157068 18366 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:46066:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 11:23:01.158538 18366 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:23:01.166839 18512 tablet_service.cc:1467] Processing CreateTablet for tablet ebc510d2643940ca98a89fbcba06c653 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=251e2133a46e4bafacd737f449e719f3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 11:23:01.167488 18512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebc510d2643940ca98a89fbcba06c653. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:01.167333 18658 tablet_service.cc:1467] Processing CreateTablet for tablet ebc510d2643940ca98a89fbcba06c653 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=251e2133a46e4bafacd737f449e719f3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 11:23:01.167616 18804 tablet_service.cc:1467] Processing CreateTablet for tablet ebc510d2643940ca98a89fbcba06c653 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=251e2133a46e4bafacd737f449e719f3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 11:23:01.167905 18658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebc510d2643940ca98a89fbcba06c653. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:01.168484 18804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebc510d2643940ca98a89fbcba06c653. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:01.177883 18897 tablet_bootstrap.cc:492] T ebc510d2643940ca98a89fbcba06c653 P 257609186abd4d8eb658659a01aa71ab: Bootstrap starting.
I20260807 11:23:01.178972 18899 tablet_bootstrap.cc:492] T ebc510d2643940ca98a89fbcba06c653 P 88ec7cf9488d40af9c742bd902fa595b: Bootstrap starting.
I20260807 11:23:01.180719 18897 tablet_bootstrap.cc:654] T ebc510d2643940ca98a89fbcba06c653 P 257609186abd4d8eb658659a01aa71ab: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:01.181222 18898 tablet_bootstrap.cc:492] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7: Bootstrap starting.
I20260807 11:23:01.181424 18899 tablet_bootstrap.cc:654] T ebc510d2643940ca98a89fbcba06c653 P 88ec7cf9488d40af9c742bd902fa595b: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:01.184782 18898 tablet_bootstrap.cc:654] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:01.194607 18899 tablet_bootstrap.cc:492] T ebc510d2643940ca98a89fbcba06c653 P 88ec7cf9488d40af9c742bd902fa595b: No bootstrap required, opened a new log
I20260807 11:23:01.194813 18898 tablet_bootstrap.cc:492] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7: No bootstrap required, opened a new log
I20260807 11:23:01.195010 18899 ts_tablet_manager.cc:1397] T ebc510d2643940ca98a89fbcba06c653 P 88ec7cf9488d40af9c742bd902fa595b: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.001s
I20260807 11:23:01.195104 18898 ts_tablet_manager.cc:1397] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.005s
I20260807 11:23:01.196339 18899 raft_consensus.cc:348] T ebc510d2643940ca98a89fbcba06c653 P 88ec7cf9488d40af9c742bd902fa595b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } }
I20260807 11:23:01.196348 18898 raft_consensus.cc:348] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } }
I20260807 11:23:01.196694 18899 raft_consensus.cc:374] T ebc510d2643940ca98a89fbcba06c653 P 88ec7cf9488d40af9c742bd902fa595b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:01.196750 18898 raft_consensus.cc:374] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:01.196879 18898 raft_consensus.cc:729] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d57748dc8134476b31453604f247bc7, State: Initialized, Role: FOLLOWER
I20260807 11:23:01.196885 18899 raft_consensus.cc:729] T ebc510d2643940ca98a89fbcba06c653 P 88ec7cf9488d40af9c742bd902fa595b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ec7cf9488d40af9c742bd902fa595b, State: Initialized, Role: FOLLOWER
I20260807 11:23:01.197230 18898 consensus_queue.cc:260] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } }
I20260807 11:23:01.197250 18899 consensus_queue.cc:260] T ebc510d2643940ca98a89fbcba06c653 P 88ec7cf9488d40af9c742bd902fa595b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } }
I20260807 11:23:01.197896 18903 tablet_replica.cc:439] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } } }
I20260807 11:23:01.198402 18903 tablet_replica.cc:442] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 11:23:01.198637 18897 tablet_bootstrap.cc:492] T ebc510d2643940ca98a89fbcba06c653 P 257609186abd4d8eb658659a01aa71ab: No bootstrap required, opened a new log
I20260807 11:23:01.198880 18897 ts_tablet_manager.cc:1397] T ebc510d2643940ca98a89fbcba06c653 P 257609186abd4d8eb658659a01aa71ab: Time spent bootstrapping tablet: real 0.021s user 0.007s sys 0.008s
I20260807 11:23:01.199698 18898 ts_tablet_manager.cc:1428] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260807 11:23:01.200362 18899 ts_tablet_manager.cc:1428] T ebc510d2643940ca98a89fbcba06c653 P 88ec7cf9488d40af9c742bd902fa595b: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260807 11:23:01.200230 18905 tablet_replica.cc:439] T ebc510d2643940ca98a89fbcba06c653 P 88ec7cf9488d40af9c742bd902fa595b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } } }
I20260807 11:23:01.200590 18905 tablet_replica.cc:442] T ebc510d2643940ca98a89fbcba06c653 P 88ec7cf9488d40af9c742bd902fa595b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 11:23:01.201161 18897 raft_consensus.cc:348] T ebc510d2643940ca98a89fbcba06c653 P 257609186abd4d8eb658659a01aa71ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } }
I20260807 11:23:01.201463 18897 raft_consensus.cc:374] T ebc510d2643940ca98a89fbcba06c653 P 257609186abd4d8eb658659a01aa71ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:01.201570 18897 raft_consensus.cc:729] T ebc510d2643940ca98a89fbcba06c653 P 257609186abd4d8eb658659a01aa71ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 257609186abd4d8eb658659a01aa71ab, State: Initialized, Role: FOLLOWER
I20260807 11:23:01.201864 18897 consensus_queue.cc:260] T ebc510d2643940ca98a89fbcba06c653 P 257609186abd4d8eb658659a01aa71ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } }
I20260807 11:23:01.202656 18897 ts_tablet_manager.cc:1428] T ebc510d2643940ca98a89fbcba06c653 P 257609186abd4d8eb658659a01aa71ab: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260807 11:23:01.202486 18906 tablet_replica.cc:439] T ebc510d2643940ca98a89fbcba06c653 P 257609186abd4d8eb658659a01aa71ab: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } } }
I20260807 11:23:01.202893 18906 tablet_replica.cc:442] T ebc510d2643940ca98a89fbcba06c653 P 257609186abd4d8eb658659a01aa71ab: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 11:23:01.358325 18903 raft_consensus.cc:482] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:23:01.358564 18903 raft_consensus.cc:504] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } }
I20260807 11:23:01.360579 18903 leader_election.cc:288] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 257609186abd4d8eb658659a01aa71ab (127.16.197.1:35649), 88ec7cf9488d40af9c742bd902fa595b (127.16.197.2:44257)
I20260807 11:23:01.372462 18945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:01.360707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:35649 (local address 127.16.197.3:43389)
0807 11:23:01.363678 (+ 2971us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:01.363695 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:01.363892 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:01.364334 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:01.364339 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:01.364363 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:01.365006 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:01.365018 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.366588 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.366593 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:01.369461 (+ 2868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:01.369475 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.371290 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.371298 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:01.371589 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:01.371631 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 11:23:01.371746 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 11:23:01.371891 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2851,"mutex_wait_us":4,"spinlock_wait_cycles":19840,"thread_start_us":2447,"threads_started":1}
I20260807 11:23:01.373289 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:01.361184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:43389 (local address 127.16.197.1:35649)
0807 11:23:01.363056 (+ 1872us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:01.363062 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:01.363811 (+ 749us) server_negotiation.cc:407] Connection header received
0807 11:23:01.364124 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:01.364129 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:01.364207 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:01.364316 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:01.365195 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.366405 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.369651 (+ 3246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.370822 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.371013 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:01.371115 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:01.372842 (+ 1727us) server_negotiation.cc:299] Negotiation successful
0807 11:23:01.372969 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1715,"thread_start_us":1428,"threads_started":1}
I20260807 11:23:01.374171 18535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebc510d2643940ca98a89fbcba06c653" candidate_uuid: "0d57748dc8134476b31453604f247bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "257609186abd4d8eb658659a01aa71ab" is_pre_election: true
I20260807 11:23:01.374598 18535 raft_consensus.cc:2454] T ebc510d2643940ca98a89fbcba06c653 P 257609186abd4d8eb658659a01aa71ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d57748dc8134476b31453604f247bc7 in term 0.
I20260807 11:23:01.375435 18754 leader_election.cc:302] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d57748dc8134476b31453604f247bc7, 257609186abd4d8eb658659a01aa71ab; no voters:
I20260807 11:23:01.375908 18903 raft_consensus.cc:2781] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:23:01.376082 18903 raft_consensus.cc:482] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:23:01.376242 18903 raft_consensus.cc:3037] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:01.376487 18893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:01.361701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:52911 (local address 127.16.197.2:44257)
0807 11:23:01.361999 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:01.362009 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:01.364388 (+ 2379us) server_negotiation.cc:407] Connection header received
0807 11:23:01.364766 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:01.364785 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:01.364897 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:01.365035 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:01.366663 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.368503 (+ 1840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.372187 (+ 3684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.374316 (+ 2129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.374554 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:01.374627 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:01.375759 (+ 1132us) server_negotiation.cc:299] Negotiation successful
0807 11:23:01.375998 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 11:23:01.377717 18946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:01.361093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:44257 (local address 127.16.197.3:52911)
0807 11:23:01.364204 (+ 3111us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:01.364236 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:01.364565 (+ 329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:01.365343 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:01.365348 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:01.365375 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:01.366294 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:01.366311 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.368657 (+ 2346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.368662 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:01.371880 (+ 3218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:01.371898 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.375093 (+ 3195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.375103 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:01.375560 (+ 457us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:01.375597 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 11:23:01.376792 (+ 1195us) client_negotiation.cc:240] Negotiation successful
0807 11:23:01.376954 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2954,"mutex_wait_us":27,"thread_start_us":2976,"threads_started":1}
I20260807 11:23:01.379963 18680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebc510d2643940ca98a89fbcba06c653" candidate_uuid: "0d57748dc8134476b31453604f247bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ec7cf9488d40af9c742bd902fa595b" is_pre_election: true
I20260807 11:23:01.379961 18903 raft_consensus.cc:504] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } }
I20260807 11:23:01.380367 18680 raft_consensus.cc:2454] T ebc510d2643940ca98a89fbcba06c653 P 88ec7cf9488d40af9c742bd902fa595b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d57748dc8134476b31453604f247bc7 in term 0.
I20260807 11:23:01.381549 18535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebc510d2643940ca98a89fbcba06c653" candidate_uuid: "0d57748dc8134476b31453604f247bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "257609186abd4d8eb658659a01aa71ab"
I20260807 11:23:01.381856 18535 raft_consensus.cc:3037] T ebc510d2643940ca98a89fbcba06c653 P 257609186abd4d8eb658659a01aa71ab [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:01.381932 18680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebc510d2643940ca98a89fbcba06c653" candidate_uuid: "0d57748dc8134476b31453604f247bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ec7cf9488d40af9c742bd902fa595b"
I20260807 11:23:01.382282 18680 raft_consensus.cc:3037] T ebc510d2643940ca98a89fbcba06c653 P 88ec7cf9488d40af9c742bd902fa595b [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:01.385841 18535 raft_consensus.cc:2454] T ebc510d2643940ca98a89fbcba06c653 P 257609186abd4d8eb658659a01aa71ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d57748dc8134476b31453604f247bc7 in term 1.
I20260807 11:23:01.386335 18680 raft_consensus.cc:2454] T ebc510d2643940ca98a89fbcba06c653 P 88ec7cf9488d40af9c742bd902fa595b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d57748dc8134476b31453604f247bc7 in term 1.
I20260807 11:23:01.386418 18754 leader_election.cc:302] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d57748dc8134476b31453604f247bc7, 257609186abd4d8eb658659a01aa71ab; no voters:
I20260807 11:23:01.381191 18903 leader_election.cc:288] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7 [CANDIDATE]: Term 1 election: Requested vote from peers 257609186abd4d8eb658659a01aa71ab (127.16.197.1:35649), 88ec7cf9488d40af9c742bd902fa595b (127.16.197.2:44257)
I20260807 11:23:01.390187 18903 raft_consensus.cc:2781] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:01.390676 18903 raft_consensus.cc:686] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7 [term 1 LEADER]: Becoming Leader. State: Replica: 0d57748dc8134476b31453604f247bc7, State: Running, Role: LEADER
I20260807 11:23:01.391428 18903 consensus_queue.cc:237] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } }
I20260807 11:23:01.396040 18948 tablet_replica.cc:439] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7: TxnStatusTablet state changed. Reason: New leader 0d57748dc8134476b31453604f247bc7. Latest consensus state: current_term: 1 leader_uuid: "0d57748dc8134476b31453604f247bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } } }
I20260807 11:23:01.396804 18948 tablet_replica.cc:442] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7: This TxnStatusTablet replica's current role is: LEADER
I20260807 11:23:01.397645 18365 catalog_manager.cc:5495] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d57748dc8134476b31453604f247bc7 (127.16.197.3). New cstate: current_term: 1 leader_uuid: "0d57748dc8134476b31453604f247bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } health_report { overall_health: HEALTHY } } }
I20260807 11:23:01.398108 18950 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 11:23:01.398519 18950 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 11:23:01.432839 18894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:01.425913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47136 (local address 127.16.197.3:40359)
0807 11:23:01.426140 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:01.426146 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:01.426330 (+ 184us) server_negotiation.cc:407] Connection header received
0807 11:23:01.426636 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:01.426641 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:01.426828 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:01.427000 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:23:01.427399 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.428506 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.430213 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.430708 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.430859 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:01.431440 (+ 581us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:23:01.432012 (+ 572us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:23:01.432246 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:01.432378 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 11:23:01.432470 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 11:23:01.432922 18940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:01.426034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:40359 (local address 127.0.0.1:47136)
0807 11:23:01.426240 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:01.426257 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:01.426429 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:01.427038 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:01.427043 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:01.427055 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 11:23:01.427261 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:01.427270 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.428709 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.428713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:01.430064 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:01.430077 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.430889 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.430895 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:01.431138 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:01.431199 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 11:23:01.432196 (+ 997us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 11:23:01.432222 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 11:23:01.432313 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 11:23:01.432493 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 11:23:01.438336 18535 raft_consensus.cc:1261] T ebc510d2643940ca98a89fbcba06c653 P 257609186abd4d8eb658659a01aa71ab [term 1 FOLLOWER]: Refusing update from remote peer 0d57748dc8134476b31453604f247bc7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:01.439002 18680 raft_consensus.cc:1261] T ebc510d2643940ca98a89fbcba06c653 P 88ec7cf9488d40af9c742bd902fa595b [term 1 FOLLOWER]: Refusing update from remote peer 0d57748dc8134476b31453604f247bc7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:01.440167 18948 consensus_queue.cc:1035] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:01.440728 18903 consensus_queue.cc:1035] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:01.446241 18905 tablet_replica.cc:439] T ebc510d2643940ca98a89fbcba06c653 P 88ec7cf9488d40af9c742bd902fa595b: TxnStatusTablet state changed. Reason: New leader 0d57748dc8134476b31453604f247bc7. Latest consensus state: current_term: 1 leader_uuid: "0d57748dc8134476b31453604f247bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } } }
I20260807 11:23:01.446861 18905 tablet_replica.cc:442] T ebc510d2643940ca98a89fbcba06c653 P 88ec7cf9488d40af9c742bd902fa595b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 11:23:01.457724 18953 mvcc.cc:205] Tried to move back new op lower bound from 7315872896762900480 to 7315872896586256384. Current Snapshot: MvccSnapshot[applied={T|T < 7315872896762900480}]
I20260807 11:23:01.460614 18905 tablet_replica.cc:439] T ebc510d2643940ca98a89fbcba06c653 P 88ec7cf9488d40af9c742bd902fa595b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d57748dc8134476b31453604f247bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } } }
I20260807 11:23:01.460973 18905 tablet_replica.cc:442] T ebc510d2643940ca98a89fbcba06c653 P 88ec7cf9488d40af9c742bd902fa595b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 11:23:01.469018 18906 tablet_replica.cc:439] T ebc510d2643940ca98a89fbcba06c653 P 257609186abd4d8eb658659a01aa71ab: TxnStatusTablet state changed. Reason: New leader 0d57748dc8134476b31453604f247bc7. Latest consensus state: current_term: 1 leader_uuid: "0d57748dc8134476b31453604f247bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } } }
I20260807 11:23:01.469594 18906 tablet_replica.cc:442] T ebc510d2643940ca98a89fbcba06c653 P 257609186abd4d8eb658659a01aa71ab: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 11:23:01.472924 18903 tablet_replica.cc:439] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d57748dc8134476b31453604f247bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } } }
I20260807 11:23:01.473996 18903 tablet_replica.cc:442] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7: This TxnStatusTablet replica's current role is: LEADER
I20260807 11:23:01.495296 18948 tablet_replica.cc:439] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d57748dc8134476b31453604f247bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } } }
I20260807 11:23:01.495903 18948 tablet_replica.cc:442] T ebc510d2643940ca98a89fbcba06c653 P 0d57748dc8134476b31453604f247bc7: This TxnStatusTablet replica's current role is: LEADER
I20260807 11:23:01.496637 18955 mvcc.cc:205] Tried to move back new op lower bound from 7315872896762900480 to 7315872896586256384. Current Snapshot: MvccSnapshot[applied={T|T < 7315872896762900480}]
I20260807 11:23:01.503685 18906 tablet_replica.cc:439] T ebc510d2643940ca98a89fbcba06c653 P 257609186abd4d8eb658659a01aa71ab: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d57748dc8134476b31453604f247bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257609186abd4d8eb658659a01aa71ab" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35649 } } peers { permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 } } peers { permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 } } }
I20260807 11:23:01.504040 18906 tablet_replica.cc:442] T ebc510d2643940ca98a89fbcba06c653 P 257609186abd4d8eb658659a01aa71ab: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 11:23:01.530076 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:01.505734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41436 (local address 127.16.197.1:35649)
0807 11:23:01.505946 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:01.505952 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:01.516171 (+ 10219us) server_negotiation.cc:407] Connection header received
0807 11:23:01.519885 (+ 3714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:01.519891 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:01.520076 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:01.520300 (+ 224us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:23:01.522050 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.523515 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.526644 (+ 3129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.527646 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.527752 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:01.528543 (+ 791us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:23:01.529241 (+ 698us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:23:01.529354 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:01.529623 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 11:23:01.529716 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 11:23:01.530644 18906 consensus_queue.cc:1035] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d57748dc8134476b31453604f247bc7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:01.555540 18906 consensus_queue.cc:1035] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ec7cf9488d40af9c742bd902fa595b" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:01.589833 18894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:01.573874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:40715 (local address 127.16.197.3:40359)
0807 11:23:01.574067 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:01.574073 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:01.578731 (+ 4658us) server_negotiation.cc:407] Connection header received
0807 11:23:01.581174 (+ 2443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:01.581180 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:01.581289 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:01.581362 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:23:01.582301 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.583981 (+ 1680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.586491 (+ 2510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.587607 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.587757 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:01.588603 (+ 846us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:23:01.588801 (+ 198us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:23:01.588876 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:01.589145 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 11:23:01.589312 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 11:23:01.591041 18971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:01.574251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:40359 (local address 127.16.197.1:40715)
0807 11:23:01.578593 (+ 4342us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:01.578632 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:01.580954 (+ 2322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:01.581599 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:01.581604 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:01.581617 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 11:23:01.582124 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:01.582139 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.584174 (+ 2035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.584179 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:01.586297 (+ 2118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:01.586313 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.587989 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.587997 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:01.588240 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:01.588330 (+ 90us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 11:23:01.588965 (+ 635us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 11:23:01.589000 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 11:23:01.590350 (+ 1350us) client_negotiation.cc:240] Negotiation successful
0807 11:23:01.590515 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3809,"thread_start_us":3851,"threads_started":1}
I20260807 11:23:01.683531 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:01.667291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:47101 (local address 127.16.197.1:35649)
0807 11:23:01.667954 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:01.667962 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:01.670253 (+ 2291us) server_negotiation.cc:407] Connection header received
0807 11:23:01.670781 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:01.670788 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:01.670918 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:01.671016 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:23:01.675399 (+ 4383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.676949 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.680126 (+ 3177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.681105 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.681267 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:01.682311 (+ 1044us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:23:01.682491 (+ 180us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:23:01.682603 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:01.682857 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 11:23:01.682949 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497}
I20260807 11:23:01.683871 18975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:01.666659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:35649 (local address 127.16.197.3:47101)
0807 11:23:01.670140 (+ 3481us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:01.670164 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:01.670505 (+ 341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:01.671344 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:01.671348 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:01.671361 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 11:23:01.675147 (+ 3786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:01.675168 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.677074 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.677079 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:01.679854 (+ 2775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:01.679874 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.681590 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.681598 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:01.681907 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:01.682012 (+ 105us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 11:23:01.682687 (+ 675us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 11:23:01.682702 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 11:23:01.682793 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 11:23:01.683001 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3118,"thread_start_us":2775,"threads_started":1}
I20260807 11:23:01.745890 18365 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:23:01.746268 18365 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:23:01.750685 18365 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b527b68d53bf4086bc4f8b2f7627b6c5: Sending DeleteTablet for 3 replicas of tablet 9ade140daff04ea384c4dc99cd26b70d
I20260807 11:23:01.752686 18658 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ade140daff04ea384c4dc99cd26b70d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:01 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:40508
I20260807 11:23:01.752890 18804 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ade140daff04ea384c4dc99cd26b70d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:01 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:47132
I20260807 11:23:01.753396 18512 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ade140daff04ea384c4dc99cd26b70d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:01 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:41420
I20260807 11:23:01.759683 18979 tablet_replica.cc:330] T 9ade140daff04ea384c4dc99cd26b70d P 0d57748dc8134476b31453604f247bc7: stopping tablet replica
I20260807 11:23:01.760119 18978 tablet_replica.cc:330] T 9ade140daff04ea384c4dc99cd26b70d P 88ec7cf9488d40af9c742bd902fa595b: stopping tablet replica
I20260807 11:23:01.760463 18978 raft_consensus.cc:2229] T 9ade140daff04ea384c4dc99cd26b70d P 88ec7cf9488d40af9c742bd902fa595b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:01.760895 18978 raft_consensus.cc:2258] T 9ade140daff04ea384c4dc99cd26b70d P 88ec7cf9488d40af9c742bd902fa595b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:01.762253 18978 ts_tablet_manager.cc:1905] T 9ade140daff04ea384c4dc99cd26b70d P 88ec7cf9488d40af9c742bd902fa595b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:01.767807 18980 tablet_replica.cc:330] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab: stopping tablet replica
I20260807 11:23:01.768116 18980 raft_consensus.cc:2229] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab [term 1 LEADER]: Raft consensus shutting down.
I20260807 11:23:01.769162 18980 raft_consensus.cc:2258] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:01.769734 18979 raft_consensus.cc:2229] T 9ade140daff04ea384c4dc99cd26b70d P 0d57748dc8134476b31453604f247bc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:01.770380 18979 raft_consensus.cc:2258] T 9ade140daff04ea384c4dc99cd26b70d P 0d57748dc8134476b31453604f247bc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:01.771668 18980 ts_tablet_manager.cc:1905] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:01.773530 18979 ts_tablet_manager.cc:1905] T 9ade140daff04ea384c4dc99cd26b70d P 0d57748dc8134476b31453604f247bc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:01.781162 18978 ts_tablet_manager.cc:1918] T 9ade140daff04ea384c4dc99cd26b70d P 88ec7cf9488d40af9c742bd902fa595b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 11:23:01.781454 18978 log.cc:1196] T 9ade140daff04ea384c4dc99cd26b70d P 88ec7cf9488d40af9c742bd902fa595b: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-1/wal/wals/9ade140daff04ea384c4dc99cd26b70d
I20260807 11:23:01.782228 18978 ts_tablet_manager.cc:1939] T 9ade140daff04ea384c4dc99cd26b70d P 88ec7cf9488d40af9c742bd902fa595b: Deleting consensus metadata
I20260807 11:23:01.783756 18352 catalog_manager.cc:4841] TS 88ec7cf9488d40af9c742bd902fa595b (127.16.197.2:44257): tablet 9ade140daff04ea384c4dc99cd26b70d (table test-table [id=3cafb041540a42e591de6240a4bad8d5]) successfully deleted
I20260807 11:23:01.788776 18979 ts_tablet_manager.cc:1918] T 9ade140daff04ea384c4dc99cd26b70d P 0d57748dc8134476b31453604f247bc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 11:23:01.789014 18979 log.cc:1196] T 9ade140daff04ea384c4dc99cd26b70d P 0d57748dc8134476b31453604f247bc7: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-2/wal/wals/9ade140daff04ea384c4dc99cd26b70d
I20260807 11:23:01.789969 18979 ts_tablet_manager.cc:1939] T 9ade140daff04ea384c4dc99cd26b70d P 0d57748dc8134476b31453604f247bc7: Deleting consensus metadata
I20260807 11:23:01.791796 18351 catalog_manager.cc:4841] TS 0d57748dc8134476b31453604f247bc7 (127.16.197.3:40359): tablet 9ade140daff04ea384c4dc99cd26b70d (table test-table [id=3cafb041540a42e591de6240a4bad8d5]) successfully deleted
I20260807 11:23:01.800892 18980 ts_tablet_manager.cc:1918] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 11:23:01.802302 18980 log.cc:1196] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786101765888975-17172-0/minicluster-data/ts-0/wal/wals/9ade140daff04ea384c4dc99cd26b70d
I20260807 11:23:01.802947 18980 ts_tablet_manager.cc:1939] T 9ade140daff04ea384c4dc99cd26b70d P 257609186abd4d8eb658659a01aa71ab: Deleting consensus metadata
I20260807 11:23:01.804538 18350 catalog_manager.cc:4841] TS 257609186abd4d8eb658659a01aa71ab (127.16.197.1:35649): tablet 9ade140daff04ea384c4dc99cd26b70d (table test-table [id=3cafb041540a42e591de6240a4bad8d5]) successfully deleted
Aug 07 11:23:01 dist-test-slave-2w3w krb5kdc[18318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101780, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.197.1@KRBTEST.COM
I20260807 11:23:01.827147 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:01.767520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41452 (local address 127.16.197.1:35649)
0807 11:23:01.767759 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:01.767766 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:01.790533 (+ 22767us) server_negotiation.cc:407] Connection header received
0807 11:23:01.791221 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:01.791228 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:01.791324 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:01.791440 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:01.796535 (+ 5095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.797996 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.801108 (+ 3112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.802022 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.802140 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:01.811200 (+ 9060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:01.811218 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:01.811239 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:01.811290 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:01.822423 (+ 11133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:01.823847 (+ 1424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:01.823859 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:01.823862 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:01.824074 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:01.824737 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:01.824743 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:01.824746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:01.825452 (+ 706us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:01.825813 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:01.826256 (+ 443us) server_negotiation.cc:299] Negotiation successful
0807 11:23:01.826552 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260807 11:23:01.828223 18583 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41452
I20260807 11:23:01.863879 18985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:01.836400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46070 (local address 127.16.197.62:43575)
0807 11:23:01.837332 (+ 932us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:01.837339 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:01.840054 (+ 2715us) server_negotiation.cc:407] Connection header received
0807 11:23:01.840292 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:01.840298 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:01.840393 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:01.840551 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:01.843870 (+ 3319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.845556 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.848204 (+ 2648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.849493 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.849661 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:01.854551 (+ 4890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:01.854577 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:01.854583 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:01.854650 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:01.860053 (+ 5403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:01.861179 (+ 1126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:01.861186 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:01.861190 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:01.861380 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:01.861888 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:01.861893 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:01.861897 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:01.862415 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:01.862676 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:01.863106 (+ 430us) server_negotiation.cc:299] Negotiation successful
0807 11:23:01.863404 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":739,"thread_start_us":535,"threads_started":1}
W20260807 11:23:01.864627 18365 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46070
I20260807 11:23:01.867728 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 18444
I20260807 11:23:01.896806 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:01.870218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41468 (local address 127.16.197.1:35649)
0807 11:23:01.870414 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:01.870420 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:01.871529 (+ 1109us) server_negotiation.cc:407] Connection header received
0807 11:23:01.871732 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:01.871738 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:01.871853 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:01.872006 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:01.873766 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.875070 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.877743 (+ 2673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.878846 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.879012 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:01.882754 (+ 3742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:01.882761 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:01.882767 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:01.882814 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:01.889279 (+ 6465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:01.890341 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:01.890347 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:01.890351 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:01.891393 (+ 1042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:01.894283 (+ 2890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:01.894288 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:01.894291 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:01.894810 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:01.895170 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:01.895940 (+ 770us) server_negotiation.cc:299] Negotiation successful
0807 11:23:01.896252 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260807 11:23:01.897709 18583 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41468
W20260807 11:23:01.898427 17172 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 18444: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 11:23:01.898726 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 18444
I20260807 11:23:01.930164 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 18592
Aug 07 11:23:01 dist-test-slave-2w3w krb5kdc[18318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101780, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.197.2@KRBTEST.COM
I20260807 11:23:01.948148 18988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:01.931057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40536 (local address 127.16.197.2:44257)
0807 11:23:01.933211 (+ 2154us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:01.933218 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:01.933239 (+ 21us) server_negotiation.cc:407] Connection header received
0807 11:23:01.933311 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:01.933316 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:01.933418 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:01.933544 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:01.934773 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.935556 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.936792 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.937231 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.937378 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:01.941992 (+ 4614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:01.942008 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:01.942022 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:01.942081 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:01.945250 (+ 3169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:01.945973 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:01.945981 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:01.945983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:01.946121 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:01.946518 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:01.946523 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:01.946526 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:01.946892 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:01.947219 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:01.947712 (+ 493us) server_negotiation.cc:299] Negotiation successful
0807 11:23:01.947893 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2008,"thread_start_us":1876,"threads_started":1}
W20260807 11:23:01.949143 18728 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40536
W20260807 11:23:01.949929 17172 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 18592: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 11:23:01.950114 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 18592
I20260807 11:23:01.983318 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 18737
Aug 07 11:23:01 dist-test-slave-2w3w krb5kdc[18318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101780, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.197.3@KRBTEST.COM
I20260807 11:23:02.001490 18894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:01.984315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47150 (local address 127.16.197.3:40359)
0807 11:23:01.984516 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:01.984522 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:01.984545 (+ 23us) server_negotiation.cc:407] Connection header received
0807 11:23:01.984622 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:01.984639 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:01.984702 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:01.984851 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:01.986097 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.986877 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.988215 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:01.988992 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:01.989144 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:01.994877 (+ 5733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:01.994891 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:01.994907 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:01.994947 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:01.998418 (+ 3471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:01.999333 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:01.999340 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:01.999347 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:01.999511 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:01.999935 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:01.999939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:01.999941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:02.000320 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:02.000591 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:02.000957 (+ 366us) server_negotiation.cc:299] Negotiation successful
0807 11:23:02.001128 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260807 11:23:02.002274 18872 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47150
W20260807 11:23:02.002990 17172 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 18737: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 11:23:02.003162 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 18737
I20260807 11:23:02.025858 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 18333
I20260807 11:23:02.027097 18411 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:02.143931 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 18333
2026-08-07T11:23:02Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5160 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:23:02 dist-test-slave-2w3w krb5kdc[18992](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:23:02 dist-test-slave-2w3w krb5kdc[18992](info): set up 2 sockets
Aug 07 11:23:02 dist-test-slave-2w3w krb5kdc[18992](info): commencing operation
krb5kdc: starting...
W20260807 11:23:03.284760 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.011s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:23:03 dist-test-slave-2w3w krb5kdc[18992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:23:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:23:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:23:03.506737 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:42523
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:45073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:42523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:03.851254 19005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:03.851660 19005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:03.851832 19005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:03.859488 19005 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:23:03.859647 19005 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 11:23:03.859751 19005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:03.859884 19005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:03.859982 19005 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:23:03.860075 19005 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:23:03.870754 19005 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:45073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:42523
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:42523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:03.872239 19005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:03.873679 19005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:03.885963 19013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:03.885917 19015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:03.885999 19012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:03.886858 19005 server_base.cc:1034] running on GCE node
I20260807 11:23:03.887789 19005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:03.889339 19005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:03.890621 19005 hybrid_clock.cc:648] HybridClock initialized: now 1786101783890557 us; error 35 us; skew 500 ppm
I20260807 11:23:03.891062 19005 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:03 dist-test-slave-2w3w krb5kdc[18992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101783, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:03.894598 19005 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:23:03.896440 19005 webserver.cc:460] Webserver started at http://127.16.197.62:42899/ using document root <none> and password file <none>
I20260807 11:23:03.897219 19005 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:03.897364 19005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:03.897786 19005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:03.901155 19005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "7ec49b30eb064f35bb6da171d3cc2d99"
format_stamp: "Formatted at 2026-08-07 11:23:03 on dist-test-slave-2w3w"
server_key: "eb453c2d9657a70a369ea52d944876d1"
server_key_iv: "f12c37306efab39e00f9389a759b2a60"
server_key_version: "encryptionkey@0"
I20260807 11:23:03.901998 19005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "7ec49b30eb064f35bb6da171d3cc2d99"
format_stamp: "Formatted at 2026-08-07 11:23:03 on dist-test-slave-2w3w"
server_key: "eb453c2d9657a70a369ea52d944876d1"
server_key_iv: "f12c37306efab39e00f9389a759b2a60"
server_key_version: "encryptionkey@0"
I20260807 11:23:03.907559 19005 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260807 11:23:03.911350 19022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:03.912859 19005 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 11:23:03.913079 19005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "7ec49b30eb064f35bb6da171d3cc2d99"
format_stamp: "Formatted at 2026-08-07 11:23:03 on dist-test-slave-2w3w"
server_key: "eb453c2d9657a70a369ea52d944876d1"
server_key_iv: "f12c37306efab39e00f9389a759b2a60"
server_key_version: "encryptionkey@0"
I20260807 11:23:03.913337 19005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:03.937389 19005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:03.941620 19005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:03.941946 19005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:03.964897 19005 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:42523
I20260807 11:23:03.964934 19085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:42523 every 8 connection(s)
I20260807 11:23:03.966441 19005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:23:03.969396 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 19005
I20260807 11:23:03.969642 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:23:03.970418 17172 external_mini_cluster.cc:1428] Setting key c16f1607bc7d8d201cb48f07be625cfb
I20260807 11:23:03.970490 19086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:03.985708 19086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ec49b30eb064f35bb6da171d3cc2d99: Bootstrap starting.
Aug 07 11:23:03 dist-test-slave-2w3w krb5kdc[18992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:03.989838 19086 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ec49b30eb064f35bb6da171d3cc2d99: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:03.991299 19086 log.cc:824] T 00000000000000000000000000000000 P 7ec49b30eb064f35bb6da171d3cc2d99: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:03.994298 19086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ec49b30eb064f35bb6da171d3cc2d99: No bootstrap required, opened a new log
I20260807 11:23:03.995862 19090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:03.972795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38338 (local address 127.16.197.62:42523)
0807 11:23:03.973695 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:03.973707 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:03.973755 (+ 48us) server_negotiation.cc:407] Connection header received
0807 11:23:03.975457 (+ 1702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:03.975480 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:03.975885 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:03.976213 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:03.978072 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:03.979492 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:03.981312 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:03.982096 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:03.982244 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:03.987375 (+ 5131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:03.987389 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:03.987412 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:03.987478 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:03.991566 (+ 4088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:03.992386 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:03.992393 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:03.992400 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:03.992596 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:03.993006 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:03.993012 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:03.993015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:03.993579 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:03.993883 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:03.994375 (+ 492us) server_negotiation.cc:299] Negotiation successful
0807 11:23:03.994803 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":401,"threads_started":1}
I20260807 11:23:04.002624 19086 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ec49b30eb064f35bb6da171d3cc2d99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ec49b30eb064f35bb6da171d3cc2d99" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42523 } }
I20260807 11:23:04.003160 19086 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ec49b30eb064f35bb6da171d3cc2d99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:04.003271 19086 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ec49b30eb064f35bb6da171d3cc2d99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ec49b30eb064f35bb6da171d3cc2d99, State: Initialized, Role: FOLLOWER
I20260807 11:23:04.003850 19086 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ec49b30eb064f35bb6da171d3cc2d99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ec49b30eb064f35bb6da171d3cc2d99" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42523 } }
I20260807 11:23:04.004094 19086 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ec49b30eb064f35bb6da171d3cc2d99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:23:04.004227 19086 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ec49b30eb064f35bb6da171d3cc2d99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:23:04.004397 19086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ec49b30eb064f35bb6da171d3cc2d99 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:04.006695 19086 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ec49b30eb064f35bb6da171d3cc2d99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ec49b30eb064f35bb6da171d3cc2d99" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42523 } }
I20260807 11:23:04.007251 19086 leader_election.cc:302] T 00000000000000000000000000000000 P 7ec49b30eb064f35bb6da171d3cc2d99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ec49b30eb064f35bb6da171d3cc2d99; no voters:
I20260807 11:23:04.007833 19086 leader_election.cc:288] T 00000000000000000000000000000000 P 7ec49b30eb064f35bb6da171d3cc2d99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:23:04.007997 19093 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ec49b30eb064f35bb6da171d3cc2d99 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:04.008713 19093 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ec49b30eb064f35bb6da171d3cc2d99 [term 1 LEADER]: Becoming Leader. State: Replica: 7ec49b30eb064f35bb6da171d3cc2d99, State: Running, Role: LEADER
I20260807 11:23:04.009276 19093 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ec49b30eb064f35bb6da171d3cc2d99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ec49b30eb064f35bb6da171d3cc2d99" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42523 } }
I20260807 11:23:04.010027 19086 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ec49b30eb064f35bb6da171d3cc2d99 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:23:04.012389 19094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ec49b30eb064f35bb6da171d3cc2d99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ec49b30eb064f35bb6da171d3cc2d99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ec49b30eb064f35bb6da171d3cc2d99" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42523 } } }
I20260807 11:23:04.012786 19094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ec49b30eb064f35bb6da171d3cc2d99 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:04.012506 19095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ec49b30eb064f35bb6da171d3cc2d99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ec49b30eb064f35bb6da171d3cc2d99. Latest consensus state: current_term: 1 leader_uuid: "7ec49b30eb064f35bb6da171d3cc2d99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ec49b30eb064f35bb6da171d3cc2d99" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42523 } } }
I20260807 11:23:04.013154 19095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ec49b30eb064f35bb6da171d3cc2d99 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:04.017418 19098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:23:04.024631 19098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:23:04.036221 19098 catalog_manager.cc:1347] Generated new cluster ID: 9c66248af3254306907a451743f6f654
I20260807 11:23:04.036360 19098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:23:04.046340 19098 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:23:04.047701 19098 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:23:04.053531 19098 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ec49b30eb064f35bb6da171d3cc2d99: Generated new TSK 0
I20260807 11:23:04.054543 19098 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.197.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
I20260807 11:23:04.110759 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:0
--local_ip_for_outbound_sockets=127.16.197.1
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:42523
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:45073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:04.436321 19115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:04.436632 19115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:04.436810 19115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:04.444216 19115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:04.444332 19115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:04.444535 19115 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 11:23:04.444633 19115 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 11:23:04.444763 19115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:23:04.454887 19115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:45073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.197.62:42523
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:04.456284 19115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:04.457674 19115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:04.470847 19121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:04.470878 19124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:04.471163 19122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:04.471869 19115 server_base.cc:1034] running on GCE node
I20260807 11:23:04.472491 19115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:04.473425 19115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:04.474669 19115 hybrid_clock.cc:648] HybridClock initialized: now 1786101784474604 us; error 82 us; skew 500 ppm
I20260807 11:23:04.475063 19115 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:04 dist-test-slave-2w3w krb5kdc[18992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101784, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:04.478189 19115 init.cc:377] Logged in from keytab as kudu/127.16.197.1@KRBTEST.COM (short username kudu)
I20260807 11:23:04.479754 19115 webserver.cc:460] Webserver started at http://127.16.197.1:38987/ using document root <none> and password file <none>
I20260807 11:23:04.480448 19115 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:04.480597 19115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:04.480913 19115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:04.483932 19115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-0/data/instance:
uuid: "e191bf52edfd4d1ab10e61af6662ec82"
format_stamp: "Formatted at 2026-08-07 11:23:04 on dist-test-slave-2w3w"
server_key: "abe4d314c71043201040d6f8d56fa0cf"
server_key_iv: "29ffb799c2b3b627c4958f05f95e6f99"
server_key_version: "encryptionkey@0"
I20260807 11:23:04.484699 19115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance:
uuid: "e191bf52edfd4d1ab10e61af6662ec82"
format_stamp: "Formatted at 2026-08-07 11:23:04 on dist-test-slave-2w3w"
server_key: "abe4d314c71043201040d6f8d56fa0cf"
server_key_iv: "29ffb799c2b3b627c4958f05f95e6f99"
server_key_version: "encryptionkey@0"
I20260807 11:23:04.489966 19115 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 11:23:04.493484 19131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:04.494807 19115 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260807 11:23:04.495034 19115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "e191bf52edfd4d1ab10e61af6662ec82"
format_stamp: "Formatted at 2026-08-07 11:23:04 on dist-test-slave-2w3w"
server_key: "abe4d314c71043201040d6f8d56fa0cf"
server_key_iv: "29ffb799c2b3b627c4958f05f95e6f99"
server_key_version: "encryptionkey@0"
I20260807 11:23:04.495234 19115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:04.529558 19115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:04.534029 19115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:04.534327 19115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:04.537346 19115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:04.537500 19115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:04.537635 19115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:04.537729 19115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 11:23:04 dist-test-slave-2w3w krb5kdc[18992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101784, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:04.562139 19140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:04.538131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:55981 (local address 127.16.197.62:42523)
0807 11:23:04.538708 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:04.538716 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:04.540005 (+ 1289us) server_negotiation.cc:407] Connection header received
0807 11:23:04.543087 (+ 3082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:04.543095 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:04.543200 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:04.543370 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:04.545729 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:04.546823 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:04.549393 (+ 2570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:04.550026 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:04.550169 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:04.554866 (+ 4697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:04.554871 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:04.554874 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:04.554907 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:04.557944 (+ 3037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:04.559002 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:04.559011 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:04.559014 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:04.559192 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:04.559821 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:04.559828 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:04.559838 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:04.560300 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:04.560454 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:04.561503 (+ 1049us) server_negotiation.cc:299] Negotiation successful
0807 11:23:04.561747 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":327,"threads_started":1}
I20260807 11:23:04.563287 19141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:04.538848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42523 (local address 127.16.197.1:55981)
0807 11:23:04.539797 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:04.539851 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:04.542680 (+ 2829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:04.543633 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:04.543641 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:04.544454 (+ 813us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:04.545504 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:04.545537 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:04.547023 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:04.547034 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:04.549227 (+ 2193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:04.549240 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:04.550303 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:04.550312 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:04.550524 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:04.551423 (+ 899us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:04.551447 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:04.554607 (+ 3160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:04.558129 (+ 3522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:04.558142 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:04.558163 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:04.558812 (+ 649us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:04.559336 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:04.559342 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:04.559352 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:04.559632 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:04.560448 (+ 816us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:04.560485 (+ 37us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:04.561022 (+ 537us) client_negotiation.cc:769] Sending connection context
0807 11:23:04.561400 (+ 378us) client_negotiation.cc:240] Negotiation successful
0807 11:23:04.561961 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":359,"threads_started":1}
I20260807 11:23:04.583220 19140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:04.568960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:60241 (local address 127.16.197.62:42523)
0807 11:23:04.569183 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:04.569192 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:04.569861 (+ 669us) server_negotiation.cc:407] Connection header received
0807 11:23:04.570096 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:04.570102 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:04.570192 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:04.570318 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:04.571896 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:04.572952 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:04.574229 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:04.574623 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:04.574782 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:04.577121 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:04.577127 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:04.577132 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:04.577173 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:04.580153 (+ 2980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:04.581158 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:04.581166 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:04.581171 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:04.581351 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:04.581716 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:04.581722 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:04.581725 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:04.582005 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:04.582161 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:04.582573 (+ 412us) server_negotiation.cc:299] Negotiation successful
0807 11:23:04.582820 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 11:23:04.583397 19196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:04.568961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42523 (local address 127.16.197.1:60241)
0807 11:23:04.569705 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:04.569728 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:04.569912 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:04.570350 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:04.570355 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:04.571161 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:04.571742 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:04.571757 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:04.573100 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:04.573106 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:04.574087 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:04.574099 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:04.574825 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:04.574832 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:04.575083 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:04.575779 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:04.575811 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:04.576925 (+ 1114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:04.580320 (+ 3395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:04.580329 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:04.580334 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:04.581018 (+ 684us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:04.581466 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:04.581471 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:04.581475 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:04.581604 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:04.582127 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:04.582132 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:04.582323 (+ 191us) client_negotiation.cc:769] Sending connection context
0807 11:23:04.582468 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 11:23:04.582823 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":388,"threads_started":1}
I20260807 11:23:04.595657 19115 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:46495
I20260807 11:23:04.595695 19259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:46495 every 8 connection(s)
I20260807 11:23:04.596931 19115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:23:04.599135 19260 heartbeater.cc:347] Connected to a master server at 127.16.197.62:42523
I20260807 11:23:04.599408 19260 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:04.600183 19260 heartbeater.cc:511] Master 127.16.197.62:42523 requested a full tablet report, sending...
I20260807 11:23:04.601915 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 19115
I20260807 11:23:04.602140 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance
I20260807 11:23:04.602159 19040 ts_manager.cc:194] Registered new tserver with Master: e191bf52edfd4d1ab10e61af6662ec82 (127.16.197.1:46495)
I20260807 11:23:04.602806 17172 external_mini_cluster.cc:1428] Setting key 81cef93eed3a690a3a6afcd2ff458ae5
I20260807 11:23:04.603901 19040 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.1@KRBTEST.COM'} at 127.16.197.1:55981
WARNING: no policy specified for kudu/127.16.197.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
I20260807 11:23:04.646611 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:0
--local_ip_for_outbound_sockets=127.16.197.2
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:42523
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:45073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:04.975642 19264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:04.975981 19264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:04.976166 19264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:04.983669 19264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:04.983851 19264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:04.983999 19264 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 11:23:04.984089 19264 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 11:23:04.984225 19264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:23:04.994570 19264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:45073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.197.62:42523
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:04.996109 19264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:04.997555 19264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:05.011897 19269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:05.011946 19272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:05.011981 19270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:05.012809 19264 server_base.cc:1034] running on GCE node
I20260807 11:23:05.013477 19264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:05.014506 19264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:05.015736 19264 hybrid_clock.cc:648] HybridClock initialized: now 1786101785015670 us; error 43 us; skew 500 ppm
I20260807 11:23:05.016054 19264 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:05 dist-test-slave-2w3w krb5kdc[18992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101785, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:05.019574 19264 init.cc:377] Logged in from keytab as kudu/127.16.197.2@KRBTEST.COM (short username kudu)
I20260807 11:23:05.021278 19264 webserver.cc:460] Webserver started at http://127.16.197.2:42807/ using document root <none> and password file <none>
I20260807 11:23:05.021998 19264 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:05.022150 19264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:05.022490 19264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:05.025620 19264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-1/data/instance:
uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc"
format_stamp: "Formatted at 2026-08-07 11:23:05 on dist-test-slave-2w3w"
server_key: "4b910fc87392a051034f3d6e955988dc"
server_key_iv: "a743037661e3ff171c493eb7a5cbc40a"
server_key_version: "encryptionkey@0"
I20260807 11:23:05.026428 19264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance:
uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc"
format_stamp: "Formatted at 2026-08-07 11:23:05 on dist-test-slave-2w3w"
server_key: "4b910fc87392a051034f3d6e955988dc"
server_key_iv: "a743037661e3ff171c493eb7a5cbc40a"
server_key_version: "encryptionkey@0"
I20260807 11:23:05.032104 19264 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 11:23:05.036048 19279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:05.037456 19264 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 11:23:05.037658 19264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc"
format_stamp: "Formatted at 2026-08-07 11:23:05 on dist-test-slave-2w3w"
server_key: "4b910fc87392a051034f3d6e955988dc"
server_key_iv: "a743037661e3ff171c493eb7a5cbc40a"
server_key_version: "encryptionkey@0"
I20260807 11:23:05.037894 19264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:05.057516 19264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:05.062175 19264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:05.062494 19264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:05.065748 19264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:05.065894 19264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:05.066113 19264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:05.066226 19264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 11:23:05 dist-test-slave-2w3w krb5kdc[18992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101785, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:05.091377 19288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:05.066824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42523 (local address 127.16.197.2:33797)
0807 11:23:05.067790 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:05.067854 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:05.069786 (+ 1932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:05.070704 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:05.070716 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:05.071676 (+ 960us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:05.072808 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:05.072849 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.074344 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.074348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:05.076380 (+ 2032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:05.076394 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.077275 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.077285 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:05.077548 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:05.078366 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:05.078395 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:05.081779 (+ 3384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:05.085662 (+ 3883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:05.085673 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:05.085692 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:05.086146 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:05.086553 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:05.086558 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:05.086562 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:05.086775 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:05.087458 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:05.087472 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:05.088157 (+ 685us) client_negotiation.cc:769] Sending connection context
0807 11:23:05.088574 (+ 417us) client_negotiation.cc:240] Negotiation successful
0807 11:23:05.089354 (+ 780us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"thread_start_us":453,"threads_started":1}
I20260807 11:23:05.091708 19140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:05.066247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:33797 (local address 127.16.197.62:42523)
0807 11:23:05.066476 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:05.066483 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:05.068008 (+ 1525us) server_negotiation.cc:407] Connection header received
0807 11:23:05.070113 (+ 2105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:05.070121 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:05.070227 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:05.070363 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:05.073065 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.074149 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.076572 (+ 2423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.077026 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.077174 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:05.082129 (+ 4955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:05.082135 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:05.082139 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:05.082192 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:05.085498 (+ 3306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:05.086289 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:05.086301 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:05.086304 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:05.086436 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:05.086919 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:05.086923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:05.086925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:05.087285 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:05.087478 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:05.091173 (+ 3695us) server_negotiation.cc:299] Negotiation successful
0807 11:23:05.091372 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 11:23:05.117744 19140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:05.099472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:34473 (local address 127.16.197.62:42523)
0807 11:23:05.099693 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:05.099700 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:05.100552 (+ 852us) server_negotiation.cc:407] Connection header received
0807 11:23:05.100796 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:05.100802 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:05.100884 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:05.100968 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:05.102759 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.104108 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.107103 (+ 2995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.107968 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.108084 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:05.111557 (+ 3473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:05.111562 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:05.111566 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:05.111599 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:05.114511 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:05.115590 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:05.115594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:05.115597 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:05.115729 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:05.116216 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:05.116220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:05.116222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:05.116575 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:05.116686 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:05.117326 (+ 640us) server_negotiation.cc:299] Negotiation successful
0807 11:23:05.117509 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 11:23:05.118479 19347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:05.099600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42523 (local address 127.16.197.2:34473)
0807 11:23:05.100408 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:05.100431 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:05.100627 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:05.101044 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:05.101052 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:05.101950 (+ 898us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:05.102526 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:05.102550 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.104350 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.104359 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:05.106812 (+ 2453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:05.106853 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.108467 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.108478 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:05.108721 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:05.109748 (+ 1027us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:05.109782 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:05.111271 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:05.114654 (+ 3383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:05.114663 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:05.114669 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:05.115449 (+ 780us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:05.115878 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:05.115886 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:05.115892 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:05.116090 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:05.116718 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:05.116726 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:05.117024 (+ 298us) client_negotiation.cc:769] Sending connection context
0807 11:23:05.117242 (+ 218us) client_negotiation.cc:240] Negotiation successful
0807 11:23:05.117783 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":417,"threads_started":1}
I20260807 11:23:05.124779 19264 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:38143
I20260807 11:23:05.124818 19403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:38143 every 8 connection(s)
I20260807 11:23:05.126012 19264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:23:05.126367 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 19264
I20260807 11:23:05.126591 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance
I20260807 11:23:05.127342 17172 external_mini_cluster.cc:1428] Setting key 61bb25e259b88a7b29651744bf73a2f6
I20260807 11:23:05.129067 19404 heartbeater.cc:347] Connected to a master server at 127.16.197.62:42523
I20260807 11:23:05.129338 19404 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:05.130244 19404 heartbeater.cc:511] Master 127.16.197.62:42523 requested a full tablet report, sending...
I20260807 11:23:05.131927 19040 ts_manager.cc:194] Registered new tserver with Master: c7eeeb95e1d14ce8b75d19f9eb8b11fc (127.16.197.2:38143)
I20260807 11:23:05.133184 19040 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.2@KRBTEST.COM'} at 127.16.197.2:33797
WARNING: no policy specified for kudu/127.16.197.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
I20260807 11:23:05.174059 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:0
--local_ip_for_outbound_sockets=127.16.197.3
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:42523
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:45073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:05.504913 19408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:05.505187 19408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:05.505290 19408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:05.512629 19408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:05.512748 19408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:05.512832 19408 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 11:23:05.512887 19408 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 11:23:05.512948 19408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:23:05.523162 19408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:45073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.197.62:42523
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:05.524595 19408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:05.525962 19408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:05.539397 19416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:05.539521 19414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:05.539418 19413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:05.544523 19408 server_base.cc:1034] running on GCE node
I20260807 11:23:05.545248 19408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:05.546271 19408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:05.547534 19408 hybrid_clock.cc:648] HybridClock initialized: now 1786101785547493 us; error 70 us; skew 500 ppm
I20260807 11:23:05.547866 19408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:05 dist-test-slave-2w3w krb5kdc[18992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101785, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:05.551188 19408 init.cc:377] Logged in from keytab as kudu/127.16.197.3@KRBTEST.COM (short username kudu)
I20260807 11:23:05.552763 19408 webserver.cc:460] Webserver started at http://127.16.197.3:35231/ using document root <none> and password file <none>
I20260807 11:23:05.553457 19408 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:05.553561 19408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:05.553828 19408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:05.556910 19408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-2/data/instance:
uuid: "3ce3f4e3b94e4538947cf5929cabdf88"
format_stamp: "Formatted at 2026-08-07 11:23:05 on dist-test-slave-2w3w"
server_key: "1d207126f13c48eaf4d4229ba3faa7e1"
server_key_iv: "c347fb54db4b40cdb2ffe65753a9443a"
server_key_version: "encryptionkey@0"
I20260807 11:23:05.557651 19408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance:
uuid: "3ce3f4e3b94e4538947cf5929cabdf88"
format_stamp: "Formatted at 2026-08-07 11:23:05 on dist-test-slave-2w3w"
server_key: "1d207126f13c48eaf4d4229ba3faa7e1"
server_key_iv: "c347fb54db4b40cdb2ffe65753a9443a"
server_key_version: "encryptionkey@0"
I20260807 11:23:05.562870 19408 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 11:23:05.566596 19423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:05.568037 19408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:23:05.568215 19408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "3ce3f4e3b94e4538947cf5929cabdf88"
format_stamp: "Formatted at 2026-08-07 11:23:05 on dist-test-slave-2w3w"
server_key: "1d207126f13c48eaf4d4229ba3faa7e1"
server_key_iv: "c347fb54db4b40cdb2ffe65753a9443a"
server_key_version: "encryptionkey@0"
I20260807 11:23:05.568442 19408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:05.593761 19408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:05.598340 19408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:05.598652 19408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:05.601491 19408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:05.601639 19408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:05.601774 19408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:05.601892 19408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:05.606546 19260 heartbeater.cc:503] Master 127.16.197.62:42523 was elected leader, sending a full tablet report...
Aug 07 11:23:05 dist-test-slave-2w3w krb5kdc[18992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101785, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:05.629089 19140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:05.602574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:54583 (local address 127.16.197.62:42523)
0807 11:23:05.602776 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:05.602784 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:05.604377 (+ 1593us) server_negotiation.cc:407] Connection header received
0807 11:23:05.607159 (+ 2782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:05.607165 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:05.607260 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:05.607350 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:05.609823 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.611131 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.614562 (+ 3431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.615213 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.615323 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:05.620775 (+ 5452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:05.620782 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:05.620787 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:05.620838 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:05.624884 (+ 4046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:05.626009 (+ 1125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:05.626016 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:05.626021 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:05.626214 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:05.626841 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:05.626850 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:05.626856 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:05.627276 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:05.627445 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:05.628451 (+ 1006us) server_negotiation.cc:299] Negotiation successful
0807 11:23:05.628701 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 11:23:05.630950 19432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:05.603327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42523 (local address 127.16.197.3:54583)
0807 11:23:05.604188 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:05.604239 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:05.606763 (+ 2524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:05.607806 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:05.607813 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:05.608311 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:05.609527 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:05.609566 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.611325 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.611334 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:05.614359 (+ 3025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:05.614378 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.615500 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.615514 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:05.615895 (+ 381us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:05.616937 (+ 1042us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:05.616985 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:05.620449 (+ 3464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:05.625051 (+ 4602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:05.625072 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:05.625094 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:05.625858 (+ 764us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:05.626363 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:05.626370 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:05.626380 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:05.626636 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:05.627561 (+ 925us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:05.627571 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:05.628018 (+ 447us) client_negotiation.cc:769] Sending connection context
0807 11:23:05.628333 (+ 315us) client_negotiation.cc:240] Negotiation successful
0807 11:23:05.628952 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":400,"threads_started":1}
I20260807 11:23:05.647614 19408 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:34011
I20260807 11:23:05.647631 19549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:34011 every 8 connection(s)
I20260807 11:23:05.649288 19408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:23:05.651691 19550 heartbeater.cc:347] Connected to a master server at 127.16.197.62:42523
I20260807 11:23:05.651981 19550 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:05.652807 19550 heartbeater.cc:511] Master 127.16.197.62:42523 requested a full tablet report, sending...
I20260807 11:23:05.654381 19040 ts_manager.cc:194] Registered new tserver with Master: 3ce3f4e3b94e4538947cf5929cabdf88 (127.16.197.3:34011)
I20260807 11:23:05.654443 19140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:05.637969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:52607 (local address 127.16.197.62:42523)
0807 11:23:05.638179 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:05.638188 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:05.639042 (+ 854us) server_negotiation.cc:407] Connection header received
0807 11:23:05.639284 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:05.639288 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:05.639368 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:05.639492 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:05.640852 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.641967 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.644778 (+ 2811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.645310 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.645412 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:05.648033 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:05.648039 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:05.648043 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:05.648081 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:05.651496 (+ 3415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:05.652436 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:05.652440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:05.652442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:05.652558 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:05.653016 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:05.653022 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:05.653025 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:05.653373 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:05.653528 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:05.653976 (+ 448us) server_negotiation.cc:299] Negotiation successful
0807 11:23:05.654182 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 11:23:05.654528 19523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:05.638055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42523 (local address 127.16.197.3:52607)
0807 11:23:05.638912 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:05.638937 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:05.639148 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:05.639531 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:05.639536 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:05.640134 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:05.640618 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:05.640639 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.642173 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.642182 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:05.644593 (+ 2411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:05.644618 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.645496 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.645502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:05.645647 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:05.646355 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:05.646382 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:05.647799 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:05.651646 (+ 3847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:05.651654 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:05.651660 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:05.652293 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:05.652672 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:05.652682 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:05.652686 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:05.652860 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:05.653493 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:05.653497 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:05.653688 (+ 191us) client_negotiation.cc:769] Sending connection context
0807 11:23:05.653828 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 11:23:05.654114 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":441,"threads_started":1}
I20260807 11:23:05.655324 19040 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.3@KRBTEST.COM'} at 127.16.197.3:54583
I20260807 11:23:05.656373 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 19408
I20260807 11:23:05.656553 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance
I20260807 11:23:05.656962 17172 external_mini_cluster.cc:1428] Setting key 370a5b0cdb1662c0defe08b189d08dcb
I20260807 11:23:05.659917 17172 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 11:23:05 dist-test-slave-2w3w krb5kdc[18992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 11:23:05 dist-test-slave-2w3w krb5kdc[18992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:05.713943 19140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:05.698680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38344 (local address 127.16.197.62:42523)
0807 11:23:05.699030 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:05.699038 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:05.699213 (+ 175us) server_negotiation.cc:407] Connection header received
0807 11:23:05.699490 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:05.699494 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:05.699594 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:05.699735 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:05.701316 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.702159 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.703678 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.704166 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.704317 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:05.708459 (+ 4142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:05.708465 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:05.708468 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:05.708504 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:05.711376 (+ 2872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:05.712107 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:05.712112 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:05.712114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:05.712265 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:05.712592 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:05.712595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:05.712597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:05.712875 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:05.713062 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:05.713347 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 11:23:05.713521 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167}
I20260807 11:23:05.718540 19040 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 11:23:05.725533 19040 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:23:05.767408 19564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:05.738355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52530 (local address 127.16.197.1:46495)
0807 11:23:05.748788 (+ 10433us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:05.748801 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:05.756565 (+ 7764us) server_negotiation.cc:407] Connection header received
0807 11:23:05.757267 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:05.757280 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:05.757605 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:05.757852 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:05.759131 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.760913 (+ 1782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.763780 (+ 2867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.765045 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.765196 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:05.765433 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:05.766887 (+ 1454us) server_negotiation.cc:299] Negotiation successful
0807 11:23:05.767108 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10215,"thread_start_us":10043,"threads_started":1}
I20260807 11:23:05.767633 19565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:05.738110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:46495 (local address 127.0.0.1:52530)
0807 11:23:05.756270 (+ 18160us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:05.756336 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:05.756925 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:05.758202 (+ 1277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:05.758213 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:05.758264 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:05.758846 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:05.758863 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.761119 (+ 2256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.761124 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:05.763617 (+ 2493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:05.763628 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.765595 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.765602 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:05.765860 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:05.765935 (+ 75us) client_negotiation.cc:769] Sending connection context
0807 11:23:05.766066 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 11:23:05.766273 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17729,"mutex_wait_us":4,"thread_start_us":22848,"threads_started":1}
I20260807 11:23:05.769649 19566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:05.739264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:38143 (local address 127.0.0.1:41058)
0807 11:23:05.760428 (+ 21164us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:05.760455 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:05.760638 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:05.761747 (+ 1109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:05.761752 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:05.761780 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:05.762348 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:05.762361 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.764422 (+ 2061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.764428 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:05.766797 (+ 2369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:05.766818 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.768897 (+ 2079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.768903 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:05.769038 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:05.769060 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 11:23:05.769169 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 11:23:05.769290 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20985,"thread_start_us":16389,"threads_started":1}
I20260807 11:23:05.769686 19567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:05.739840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41058 (local address 127.16.197.2:38143)
0807 11:23:05.755159 (+ 15319us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:05.755173 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:05.760607 (+ 5434us) server_negotiation.cc:407] Connection header received
0807 11:23:05.760824 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:05.760835 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:05.761123 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:05.761369 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:05.762582 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.764254 (+ 1672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.767023 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.768392 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.768497 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:05.768700 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:05.769214 (+ 514us) server_negotiation.cc:299] Negotiation successful
0807 11:23:05.769396 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15062,"thread_start_us":15331,"threads_started":1}
I20260807 11:23:05.772621 19568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:05.740596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53986 (local address 127.16.197.3:34011)
0807 11:23:05.759793 (+ 19197us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:05.759806 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:05.761420 (+ 1614us) server_negotiation.cc:407] Connection header received
0807 11:23:05.761736 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:05.761745 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:05.761984 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:05.762174 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:05.763206 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.764874 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.767525 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.769110 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.769258 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:05.769509 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:05.771531 (+ 2022us) server_negotiation.cc:299] Negotiation successful
0807 11:23:05.772147 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18935,"thread_start_us":18514,"threads_started":1}
I20260807 11:23:05.772294 19185 tablet_service.cc:1467] Processing CreateTablet for tablet 1762723c861344e883dcafa9b772a4b2 (DEFAULT_TABLE table=test-table [id=e369eabb47574107ae129e059d967458]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:05.773458 19563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:05.739859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:34011 (local address 127.0.0.1:53986)
0807 11:23:05.761258 (+ 21399us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:05.761316 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:05.761541 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:05.762501 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:05.762509 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:05.762543 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:05.762947 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:05.762982 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.765013 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.765020 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:05.767364 (+ 2344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:05.767377 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.769683 (+ 2306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.769690 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:05.770896 (+ 1206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:05.770934 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 11:23:05.771047 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 11:23:05.773061 (+ 2014us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21247,"mutex_wait_us":1,"thread_start_us":20141,"threads_started":1}
I20260807 11:23:05.773231 19332 tablet_service.cc:1467] Processing CreateTablet for tablet 1762723c861344e883dcafa9b772a4b2 (DEFAULT_TABLE table=test-table [id=e369eabb47574107ae129e059d967458]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:05.773999 19185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1762723c861344e883dcafa9b772a4b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:05.774540 19332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1762723c861344e883dcafa9b772a4b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:05.777385 19476 tablet_service.cc:1467] Processing CreateTablet for tablet 1762723c861344e883dcafa9b772a4b2 (DEFAULT_TABLE table=test-table [id=e369eabb47574107ae129e059d967458]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:05.779309 19476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1762723c861344e883dcafa9b772a4b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:05.786540 19569 tablet_bootstrap.cc:492] T 1762723c861344e883dcafa9b772a4b2 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Bootstrap starting.
I20260807 11:23:05.789705 19569 tablet_bootstrap.cc:654] T 1762723c861344e883dcafa9b772a4b2 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:05.790439 19570 tablet_bootstrap.cc:492] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82: Bootstrap starting.
I20260807 11:23:05.791357 19569 log.cc:824] T 1762723c861344e883dcafa9b772a4b2 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:05.793946 19570 tablet_bootstrap.cc:654] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:05.794739 19569 tablet_bootstrap.cc:492] T 1762723c861344e883dcafa9b772a4b2 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: No bootstrap required, opened a new log
I20260807 11:23:05.795058 19569 ts_tablet_manager.cc:1397] T 1762723c861344e883dcafa9b772a4b2 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260807 11:23:05.795274 19570 log.cc:824] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:05.798513 19570 tablet_bootstrap.cc:492] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82: No bootstrap required, opened a new log
I20260807 11:23:05.796707 19572 tablet_bootstrap.cc:492] T 1762723c861344e883dcafa9b772a4b2 P 3ce3f4e3b94e4538947cf5929cabdf88: Bootstrap starting.
I20260807 11:23:05.798812 19570 ts_tablet_manager.cc:1397] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260807 11:23:05.802120 19572 tablet_bootstrap.cc:654] T 1762723c861344e883dcafa9b772a4b2 P 3ce3f4e3b94e4538947cf5929cabdf88: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:05.803402 19572 log.cc:824] T 1762723c861344e883dcafa9b772a4b2 P 3ce3f4e3b94e4538947cf5929cabdf88: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:05.806005 19569 raft_consensus.cc:348] T 1762723c861344e883dcafa9b772a4b2 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:05.806468 19569 raft_consensus.cc:374] T 1762723c861344e883dcafa9b772a4b2 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:05.806574 19569 raft_consensus.cc:729] T 1762723c861344e883dcafa9b772a4b2 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7eeeb95e1d14ce8b75d19f9eb8b11fc, State: Initialized, Role: FOLLOWER
I20260807 11:23:05.807230 19572 tablet_bootstrap.cc:492] T 1762723c861344e883dcafa9b772a4b2 P 3ce3f4e3b94e4538947cf5929cabdf88: No bootstrap required, opened a new log
I20260807 11:23:05.807268 19569 consensus_queue.cc:260] T 1762723c861344e883dcafa9b772a4b2 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:05.807561 19572 ts_tablet_manager.cc:1397] T 1762723c861344e883dcafa9b772a4b2 P 3ce3f4e3b94e4538947cf5929cabdf88: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260807 11:23:05.808446 19404 heartbeater.cc:503] Master 127.16.197.62:42523 was elected leader, sending a full tablet report...
I20260807 11:23:05.809602 19570 raft_consensus.cc:348] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:05.810056 19570 raft_consensus.cc:374] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:05.810160 19570 raft_consensus.cc:729] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e191bf52edfd4d1ab10e61af6662ec82, State: Initialized, Role: FOLLOWER
I20260807 11:23:05.810639 19569 ts_tablet_manager.cc:1428] T 1762723c861344e883dcafa9b772a4b2 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Time spent starting tablet: real 0.015s user 0.009s sys 0.004s
I20260807 11:23:05.810859 19570 consensus_queue.cc:260] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:05.812726 19570 ts_tablet_manager.cc:1428] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82: Time spent starting tablet: real 0.014s user 0.010s sys 0.003s
I20260807 11:23:05.818595 19572 raft_consensus.cc:348] T 1762723c861344e883dcafa9b772a4b2 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:05.819139 19572 raft_consensus.cc:374] T 1762723c861344e883dcafa9b772a4b2 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:05.819255 19572 raft_consensus.cc:729] T 1762723c861344e883dcafa9b772a4b2 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce3f4e3b94e4538947cf5929cabdf88, State: Initialized, Role: FOLLOWER
I20260807 11:23:05.819828 19572 consensus_queue.cc:260] T 1762723c861344e883dcafa9b772a4b2 P 3ce3f4e3b94e4538947cf5929cabdf88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:05.820855 19550 heartbeater.cc:503] Master 127.16.197.62:42523 was elected leader, sending a full tablet report...
I20260807 11:23:05.821624 19572 ts_tablet_manager.cc:1428] T 1762723c861344e883dcafa9b772a4b2 P 3ce3f4e3b94e4538947cf5929cabdf88: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260807 11:23:05.849388 19261 tablet.cc:2369] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 11:23:05.882720 19405 tablet.cc:2369] T 1762723c861344e883dcafa9b772a4b2 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:23:05.889400 19576 raft_consensus.cc:482] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:23:05.889832 19576 raft_consensus.cc:504] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:05.891795 19576 leader_election.cc:288] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7eeeb95e1d14ce8b75d19f9eb8b11fc (127.16.197.2:38143), 3ce3f4e3b94e4538947cf5929cabdf88 (127.16.197.3:34011)
I20260807 11:23:05.899513 19567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:05.892213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:37493 (local address 127.16.197.2:38143)
0807 11:23:05.892483 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:05.892494 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:05.893409 (+ 915us) server_negotiation.cc:407] Connection header received
0807 11:23:05.893620 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:05.893624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:05.893773 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:05.893876 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:05.894658 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.895576 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.897540 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.898415 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.898549 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:05.898606 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:05.899058 (+ 452us) server_negotiation.cc:299] Negotiation successful
0807 11:23:05.899215 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 11:23:05.899659 19568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:05.892491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:57815 (local address 127.16.197.3:34011)
0807 11:23:05.892760 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:05.892766 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:05.893349 (+ 583us) server_negotiation.cc:407] Connection header received
0807 11:23:05.893590 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:05.893594 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:05.893654 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:05.893792 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:05.894679 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.895713 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.897623 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.898612 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.898789 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:05.898864 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:05.899269 (+ 405us) server_negotiation.cc:299] Negotiation successful
0807 11:23:05.899422 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 11:23:05.899700 19579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:05.892155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:38143 (local address 127.16.197.1:37493)
0807 11:23:05.893271 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:05.893291 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:05.893469 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:05.893911 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:05.893915 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:05.893944 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:05.894487 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:05.894498 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.895712 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.895717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:05.897348 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:05.897365 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.898633 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.898641 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:05.898823 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:05.898861 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 11:23:05.899048 (+ 187us) client_negotiation.cc:240] Negotiation successful
0807 11:23:05.899212 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1013,"mutex_wait_us":19,"thread_start_us":541,"threads_started":1}
I20260807 11:23:05.900187 19580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:05.892123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:34011 (local address 127.16.197.1:57815)
0807 11:23:05.893175 (+ 1052us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:05.893201 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:05.893437 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:05.893833 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:05.893884 (+ 51us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:05.893922 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:05.894506 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:05.894517 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.895884 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.895888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:05.897405 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:05.897418 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.898878 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.898883 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:05.899089 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:05.899127 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 11:23:05.899220 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 11:23:05.899347 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":841,"mutex_wait_us":26,"thread_start_us":889,"threads_started":1}
I20260807 11:23:05.900895 19358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1762723c861344e883dcafa9b772a4b2" candidate_uuid: "e191bf52edfd4d1ab10e61af6662ec82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" is_pre_election: true
W20260807 11:23:05.901352 19551 tablet.cc:2369] T 1762723c861344e883dcafa9b772a4b2 P 3ce3f4e3b94e4538947cf5929cabdf88: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:23:05.901512 19358 raft_consensus.cc:2454] T 1762723c861344e883dcafa9b772a4b2 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e191bf52edfd4d1ab10e61af6662ec82 in term 0.
I20260807 11:23:05.901372 19498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1762723c861344e883dcafa9b772a4b2" candidate_uuid: "e191bf52edfd4d1ab10e61af6662ec82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" is_pre_election: true
I20260807 11:23:05.902021 19498 raft_consensus.cc:2454] T 1762723c861344e883dcafa9b772a4b2 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e191bf52edfd4d1ab10e61af6662ec82 in term 0.
I20260807 11:23:05.902268 19134 leader_election.cc:302] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7eeeb95e1d14ce8b75d19f9eb8b11fc, e191bf52edfd4d1ab10e61af6662ec82; no voters:
I20260807 11:23:05.902745 19576 raft_consensus.cc:2781] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:23:05.902909 19576 raft_consensus.cc:482] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:23:05.903059 19576 raft_consensus.cc:3037] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:05.905304 19576 raft_consensus.cc:504] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:05.905939 19576 leader_election.cc:288] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82 [CANDIDATE]: Term 1 election: Requested vote from peers c7eeeb95e1d14ce8b75d19f9eb8b11fc (127.16.197.2:38143), 3ce3f4e3b94e4538947cf5929cabdf88 (127.16.197.3:34011)
I20260807 11:23:05.906411 19358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1762723c861344e883dcafa9b772a4b2" candidate_uuid: "e191bf52edfd4d1ab10e61af6662ec82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc"
I20260807 11:23:05.906517 19498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1762723c861344e883dcafa9b772a4b2" candidate_uuid: "e191bf52edfd4d1ab10e61af6662ec82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce3f4e3b94e4538947cf5929cabdf88"
I20260807 11:23:05.906649 19358 raft_consensus.cc:3037] T 1762723c861344e883dcafa9b772a4b2 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:05.906721 19498 raft_consensus.cc:3037] T 1762723c861344e883dcafa9b772a4b2 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:05.910034 19358 raft_consensus.cc:2454] T 1762723c861344e883dcafa9b772a4b2 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e191bf52edfd4d1ab10e61af6662ec82 in term 1.
I20260807 11:23:05.910089 19498 raft_consensus.cc:2454] T 1762723c861344e883dcafa9b772a4b2 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e191bf52edfd4d1ab10e61af6662ec82 in term 1.
I20260807 11:23:05.910543 19135 leader_election.cc:302] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce3f4e3b94e4538947cf5929cabdf88, e191bf52edfd4d1ab10e61af6662ec82; no voters:
I20260807 11:23:05.910934 19576 raft_consensus.cc:2781] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:05.911515 19576 raft_consensus.cc:686] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82 [term 1 LEADER]: Becoming Leader. State: Replica: e191bf52edfd4d1ab10e61af6662ec82, State: Running, Role: LEADER
I20260807 11:23:05.912101 19576 consensus_queue.cc:237] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:05.916790 19039 catalog_manager.cc:5495] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82 reported cstate change: term changed from 0 to 1, leader changed from <none> to e191bf52edfd4d1ab10e61af6662ec82 (127.16.197.1). New cstate: current_term: 1 leader_uuid: "e191bf52edfd4d1ab10e61af6662ec82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } health_report { overall_health: UNKNOWN } } }
Aug 07 11:23:05 dist-test-slave-2w3w krb5kdc[18992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101783, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:06.003156 19140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:05.985452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38356 (local address 127.16.197.62:42523)
0807 11:23:05.985671 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:05.985675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:05.986243 (+ 568us) server_negotiation.cc:407] Connection header received
0807 11:23:05.986371 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:05.986374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:05.986444 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:05.986530 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:05.987841 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.988610 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.990610 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.991485 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.991657 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:05.996314 (+ 4657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:05.996320 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:05.996326 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:05.996362 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:06.000208 (+ 3846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:06.000988 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:06.000995 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:06.000998 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:06.001137 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:06.001538 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:06.001545 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:06.001548 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:06.001846 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:06.002044 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:06.002453 (+ 409us) server_negotiation.cc:299] Negotiation successful
0807 11:23:06.002800 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":159,"server-negotiator.queue_time_us":64}
I20260807 11:23:06.003193 19589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:05.985470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42523 (local address 127.0.0.1:38356)
0807 11:23:05.986110 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:05.986130 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:05.986261 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:05.986584 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:05.986587 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:05.987422 (+ 835us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:05.987687 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:05.987696 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.988801 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.988812 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:05.990443 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:05.990463 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:05.991734 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:05.991743 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:05.991982 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:05.992781 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:05.992809 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:05.996116 (+ 3307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:06.000369 (+ 4253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:06.000383 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:06.000391 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:06.000865 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:06.001246 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:06.001252 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:06.001258 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:06.001430 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:06.001979 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:06.001991 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:06.002180 (+ 189us) client_negotiation.cc:769] Sending connection context
0807 11:23:06.002361 (+ 181us) client_negotiation.cc:240] Negotiation successful
0807 11:23:06.002678 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":381,"threads_started":1}
I20260807 11:23:06.010133 19039 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:38356:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 11:23:06.011559 19039 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:23:06.020042 19185 tablet_service.cc:1467] Processing CreateTablet for tablet 77fb0d40c7fd429398cdc88f6e6bb5bf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ecdc79de3e42464f855f51c7b894c9ee]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 11:23:06.020457 19476 tablet_service.cc:1467] Processing CreateTablet for tablet 77fb0d40c7fd429398cdc88f6e6bb5bf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ecdc79de3e42464f855f51c7b894c9ee]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 11:23:06.021174 19185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77fb0d40c7fd429398cdc88f6e6bb5bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:06.020668 19332 tablet_service.cc:1467] Processing CreateTablet for tablet 77fb0d40c7fd429398cdc88f6e6bb5bf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ecdc79de3e42464f855f51c7b894c9ee]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 11:23:06.021301 19476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77fb0d40c7fd429398cdc88f6e6bb5bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:06.021589 19332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77fb0d40c7fd429398cdc88f6e6bb5bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:06.027045 19569 tablet_bootstrap.cc:492] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Bootstrap starting.
I20260807 11:23:06.028543 19572 tablet_bootstrap.cc:492] T 77fb0d40c7fd429398cdc88f6e6bb5bf P 3ce3f4e3b94e4538947cf5929cabdf88: Bootstrap starting.
I20260807 11:23:06.028577 19570 tablet_bootstrap.cc:492] T 77fb0d40c7fd429398cdc88f6e6bb5bf P e191bf52edfd4d1ab10e61af6662ec82: Bootstrap starting.
I20260807 11:23:06.029004 19569 tablet_bootstrap.cc:654] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:06.030007 19572 tablet_bootstrap.cc:654] T 77fb0d40c7fd429398cdc88f6e6bb5bf P 3ce3f4e3b94e4538947cf5929cabdf88: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:06.030292 19570 tablet_bootstrap.cc:654] T 77fb0d40c7fd429398cdc88f6e6bb5bf P e191bf52edfd4d1ab10e61af6662ec82: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:06.031669 19569 tablet_bootstrap.cc:492] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc: No bootstrap required, opened a new log
I20260807 11:23:06.031873 19569 ts_tablet_manager.cc:1397] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 11:23:06.032459 19572 tablet_bootstrap.cc:492] T 77fb0d40c7fd429398cdc88f6e6bb5bf P 3ce3f4e3b94e4538947cf5929cabdf88: No bootstrap required, opened a new log
I20260807 11:23:06.032614 19572 ts_tablet_manager.cc:1397] T 77fb0d40c7fd429398cdc88f6e6bb5bf P 3ce3f4e3b94e4538947cf5929cabdf88: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260807 11:23:06.033028 19569 raft_consensus.cc:348] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:06.033416 19569 raft_consensus.cc:374] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:06.033293 19572 raft_consensus.cc:348] T 77fb0d40c7fd429398cdc88f6e6bb5bf P 3ce3f4e3b94e4538947cf5929cabdf88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:06.033545 19569 raft_consensus.cc:729] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7eeeb95e1d14ce8b75d19f9eb8b11fc, State: Initialized, Role: FOLLOWER
I20260807 11:23:06.033609 19572 raft_consensus.cc:374] T 77fb0d40c7fd429398cdc88f6e6bb5bf P 3ce3f4e3b94e4538947cf5929cabdf88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:06.033716 19572 raft_consensus.cc:729] T 77fb0d40c7fd429398cdc88f6e6bb5bf P 3ce3f4e3b94e4538947cf5929cabdf88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce3f4e3b94e4538947cf5929cabdf88, State: Initialized, Role: FOLLOWER
I20260807 11:23:06.033838 19569 consensus_queue.cc:260] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:06.033987 19572 consensus_queue.cc:260] T 77fb0d40c7fd429398cdc88f6e6bb5bf P 3ce3f4e3b94e4538947cf5929cabdf88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:06.034413 19578 tablet_replica.cc:439] T 77fb0d40c7fd429398cdc88f6e6bb5bf P 3ce3f4e3b94e4538947cf5929cabdf88: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } } }
I20260807 11:23:06.034404 19575 tablet_replica.cc:439] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } } }
I20260807 11:23:06.034797 19578 tablet_replica.cc:442] T 77fb0d40c7fd429398cdc88f6e6bb5bf P 3ce3f4e3b94e4538947cf5929cabdf88: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 11:23:06.034797 19575 tablet_replica.cc:442] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 11:23:06.035306 19569 ts_tablet_manager.cc:1428] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 11:23:06.035444 19572 ts_tablet_manager.cc:1428] T 77fb0d40c7fd429398cdc88f6e6bb5bf P 3ce3f4e3b94e4538947cf5929cabdf88: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260807 11:23:06.047906 19570 tablet_bootstrap.cc:492] T 77fb0d40c7fd429398cdc88f6e6bb5bf P e191bf52edfd4d1ab10e61af6662ec82: No bootstrap required, opened a new log
I20260807 11:23:06.048146 19570 ts_tablet_manager.cc:1397] T 77fb0d40c7fd429398cdc88f6e6bb5bf P e191bf52edfd4d1ab10e61af6662ec82: Time spent bootstrapping tablet: real 0.020s user 0.011s sys 0.001s
I20260807 11:23:06.049197 19570 raft_consensus.cc:348] T 77fb0d40c7fd429398cdc88f6e6bb5bf P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:06.049546 19570 raft_consensus.cc:374] T 77fb0d40c7fd429398cdc88f6e6bb5bf P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:06.049655 19570 raft_consensus.cc:729] T 77fb0d40c7fd429398cdc88f6e6bb5bf P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e191bf52edfd4d1ab10e61af6662ec82, State: Initialized, Role: FOLLOWER
I20260807 11:23:06.049919 19570 consensus_queue.cc:260] T 77fb0d40c7fd429398cdc88f6e6bb5bf P e191bf52edfd4d1ab10e61af6662ec82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:06.050463 19576 tablet_replica.cc:439] T 77fb0d40c7fd429398cdc88f6e6bb5bf P e191bf52edfd4d1ab10e61af6662ec82: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } } }
I20260807 11:23:06.050822 19576 tablet_replica.cc:442] T 77fb0d40c7fd429398cdc88f6e6bb5bf P e191bf52edfd4d1ab10e61af6662ec82: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 11:23:06.052997 19570 ts_tablet_manager.cc:1428] T 77fb0d40c7fd429398cdc88f6e6bb5bf P e191bf52edfd4d1ab10e61af6662ec82: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20260807 11:23:06.281965 19575 raft_consensus.cc:482] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:23:06.282213 19575 raft_consensus.cc:504] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:06.283936 19575 leader_election.cc:288] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e191bf52edfd4d1ab10e61af6662ec82 (127.16.197.1:46495), 3ce3f4e3b94e4538947cf5929cabdf88 (127.16.197.3:34011)
I20260807 11:23:06.291191 19568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:06.284385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:44681 (local address 127.16.197.3:34011)
0807 11:23:06.284717 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:06.284726 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:06.285434 (+ 708us) server_negotiation.cc:407] Connection header received
0807 11:23:06.285765 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:06.285770 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:06.285861 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:06.285920 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:06.286661 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:06.287465 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:06.289257 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:06.290309 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:06.290482 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:06.290565 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:06.290779 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 11:23:06.290926 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160}
I20260807 11:23:06.291253 19595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:06.284204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:34011 (local address 127.16.197.2:44681)
0807 11:23:06.285313 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:06.285335 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:06.285568 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:06.286090 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:06.286093 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:06.286113 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:06.286475 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:06.286486 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:06.287604 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:06.287610 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:06.289104 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:06.289116 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:06.290511 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:06.290515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:06.290627 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:06.290652 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 11:23:06.290725 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 11:23:06.290820 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":959,"thread_start_us":1163,"threads_started":1}
I20260807 11:23:06.291863 19593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:06.284332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:46495 (local address 127.16.197.2:54191)
0807 11:23:06.285736 (+ 1404us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:06.285755 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:06.285902 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:06.286250 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:06.286260 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:06.286286 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:06.286680 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:06.286689 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:06.287855 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:06.287861 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:06.289445 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:06.289458 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:06.291069 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:06.291076 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:06.291258 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:06.291315 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 11:23:06.291410 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 11:23:06.291548 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1250,"thread_start_us":622,"threads_started":1}
I20260807 11:23:06.291877 19498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77fb0d40c7fd429398cdc88f6e6bb5bf" candidate_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" is_pre_election: true
I20260807 11:23:06.292049 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:06.284404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:54191 (local address 127.16.197.1:46495)
0807 11:23:06.285939 (+ 1535us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:06.285948 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:06.285970 (+ 22us) server_negotiation.cc:407] Connection header received
0807 11:23:06.286050 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:06.286054 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:06.286111 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:06.286226 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:06.286832 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:06.287719 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:06.289622 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:06.290804 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:06.290991 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:06.291065 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:06.291626 (+ 561us) server_negotiation.cc:299] Negotiation successful
0807 11:23:06.291764 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1399,"thread_start_us":1281,"threads_started":1}
I20260807 11:23:06.292153 19498 raft_consensus.cc:2454] T 77fb0d40c7fd429398cdc88f6e6bb5bf P 3ce3f4e3b94e4538947cf5929cabdf88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7eeeb95e1d14ce8b75d19f9eb8b11fc in term 0.
I20260807 11:23:06.292688 19283 leader_election.cc:302] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce3f4e3b94e4538947cf5929cabdf88, c7eeeb95e1d14ce8b75d19f9eb8b11fc; no voters:
I20260807 11:23:06.293025 19211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77fb0d40c7fd429398cdc88f6e6bb5bf" candidate_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e191bf52edfd4d1ab10e61af6662ec82" is_pre_election: true
I20260807 11:23:06.293196 19575 raft_consensus.cc:2781] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:23:06.293334 19575 raft_consensus.cc:482] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:23:06.293452 19575 raft_consensus.cc:3037] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:06.293444 19211 raft_consensus.cc:2454] T 77fb0d40c7fd429398cdc88f6e6bb5bf P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7eeeb95e1d14ce8b75d19f9eb8b11fc in term 0.
I20260807 11:23:06.296329 19575 raft_consensus.cc:504] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:06.297029 19575 leader_election.cc:288] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc [CANDIDATE]: Term 1 election: Requested vote from peers e191bf52edfd4d1ab10e61af6662ec82 (127.16.197.1:46495), 3ce3f4e3b94e4538947cf5929cabdf88 (127.16.197.3:34011)
I20260807 11:23:06.297433 19211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77fb0d40c7fd429398cdc88f6e6bb5bf" candidate_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e191bf52edfd4d1ab10e61af6662ec82"
I20260807 11:23:06.297658 19211 raft_consensus.cc:3037] T 77fb0d40c7fd429398cdc88f6e6bb5bf P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:06.297675 19497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77fb0d40c7fd429398cdc88f6e6bb5bf" candidate_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce3f4e3b94e4538947cf5929cabdf88"
I20260807 11:23:06.297894 19497 raft_consensus.cc:3037] T 77fb0d40c7fd429398cdc88f6e6bb5bf P 3ce3f4e3b94e4538947cf5929cabdf88 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:06.297972 19576 consensus_queue.cc:1035] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:06.300581 19497 raft_consensus.cc:2454] T 77fb0d40c7fd429398cdc88f6e6bb5bf P 3ce3f4e3b94e4538947cf5929cabdf88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7eeeb95e1d14ce8b75d19f9eb8b11fc in term 1.
I20260807 11:23:06.300685 19211 raft_consensus.cc:2454] T 77fb0d40c7fd429398cdc88f6e6bb5bf P e191bf52edfd4d1ab10e61af6662ec82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7eeeb95e1d14ce8b75d19f9eb8b11fc in term 1.
I20260807 11:23:06.301075 19283 leader_election.cc:302] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce3f4e3b94e4538947cf5929cabdf88, c7eeeb95e1d14ce8b75d19f9eb8b11fc; no voters:
I20260807 11:23:06.301501 19575 raft_consensus.cc:2781] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:06.302445 19575 raft_consensus.cc:686] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 1 LEADER]: Becoming Leader. State: Replica: c7eeeb95e1d14ce8b75d19f9eb8b11fc, State: Running, Role: LEADER
I20260807 11:23:06.302986 19575 consensus_queue.cc:237] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:06.308346 19600 consensus_queue.cc:1035] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:06.314442 19597 tablet_replica.cc:439] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc: TxnStatusTablet state changed. Reason: New leader c7eeeb95e1d14ce8b75d19f9eb8b11fc. Latest consensus state: current_term: 1 leader_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } } }
I20260807 11:23:06.314843 19597 tablet_replica.cc:442] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc: This TxnStatusTablet replica's current role is: LEADER
I20260807 11:23:06.315567 19606 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 11:23:06.315826 19606 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 11:23:06.315800 19040 catalog_manager.cc:5495] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc reported cstate change: term changed from 0 to 1, leader changed from <none> to c7eeeb95e1d14ce8b75d19f9eb8b11fc (127.16.197.2). New cstate: current_term: 1 leader_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } health_report { overall_health: UNKNOWN } } }
I20260807 11:23:06.343117 19589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:06.337294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:38143 (local address 127.0.0.1:41072)
0807 11:23:06.337545 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:06.337578 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:06.337738 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:06.338249 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:06.338254 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:06.338277 (+ 23us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 11:23:06.338472 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:06.338480 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:06.339625 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:06.339630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:06.340644 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:06.340655 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:06.341429 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:06.341435 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:06.341621 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:06.341675 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 11:23:06.342553 (+ 878us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 11:23:06.342575 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 11:23:06.342651 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 11:23:06.342803 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 11:23:06.343158 19567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:06.337345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41072 (local address 127.16.197.2:38143)
0807 11:23:06.337525 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:06.337531 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:06.337651 (+ 120us) server_negotiation.cc:407] Connection header received
0807 11:23:06.337960 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:06.337964 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:06.338043 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:06.338216 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:23:06.338638 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:06.339494 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:06.340809 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:06.341267 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:06.341413 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:06.341897 (+ 484us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:23:06.342437 (+ 540us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:23:06.342629 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:06.342677 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 11:23:06.342757 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 11:23:06.350265 19498 raft_consensus.cc:1261] T 77fb0d40c7fd429398cdc88f6e6bb5bf P 3ce3f4e3b94e4538947cf5929cabdf88 [term 1 FOLLOWER]: Refusing update from remote peer c7eeeb95e1d14ce8b75d19f9eb8b11fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:06.350713 19211 raft_consensus.cc:1261] T 77fb0d40c7fd429398cdc88f6e6bb5bf P e191bf52edfd4d1ab10e61af6662ec82 [term 1 FOLLOWER]: Refusing update from remote peer c7eeeb95e1d14ce8b75d19f9eb8b11fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:06.351189 19597 consensus_queue.cc:1035] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:06.351572 19575 consensus_queue.cc:1035] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:06.364790 19578 tablet_replica.cc:439] T 77fb0d40c7fd429398cdc88f6e6bb5bf P 3ce3f4e3b94e4538947cf5929cabdf88: TxnStatusTablet state changed. Reason: New leader c7eeeb95e1d14ce8b75d19f9eb8b11fc. Latest consensus state: current_term: 1 leader_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } } }
I20260807 11:23:06.365097 19578 tablet_replica.cc:442] T 77fb0d40c7fd429398cdc88f6e6bb5bf P 3ce3f4e3b94e4538947cf5929cabdf88: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 11:23:06.366943 19605 mvcc.cc:205] Tried to move back new op lower bound from 7315872916882206720 to 7315872916701753344. Current Snapshot: MvccSnapshot[applied={T|T < 7315872916882206720}]
I20260807 11:23:06.368022 19600 tablet_replica.cc:439] T 77fb0d40c7fd429398cdc88f6e6bb5bf P e191bf52edfd4d1ab10e61af6662ec82: TxnStatusTablet state changed. Reason: New leader c7eeeb95e1d14ce8b75d19f9eb8b11fc. Latest consensus state: current_term: 1 leader_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } } }
I20260807 11:23:06.368366 19600 tablet_replica.cc:442] T 77fb0d40c7fd429398cdc88f6e6bb5bf P e191bf52edfd4d1ab10e61af6662ec82: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 11:23:06.372897 19598 mvcc.cc:205] Tried to move back new op lower bound from 7315872916882206720 to 7315872916701753344. Current Snapshot: MvccSnapshot[applied={T|T < 7315872916882206720}]
I20260807 11:23:06.373957 19597 tablet_replica.cc:439] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } } }
I20260807 11:23:06.373894 19602 tablet_replica.cc:439] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } } }
I20260807 11:23:06.374364 19602 tablet_replica.cc:442] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc: This TxnStatusTablet replica's current role is: LEADER
I20260807 11:23:06.374503 19597 tablet_replica.cc:442] T 77fb0d40c7fd429398cdc88f6e6bb5bf P c7eeeb95e1d14ce8b75d19f9eb8b11fc: This TxnStatusTablet replica's current role is: LEADER
I20260807 11:23:06.383665 19578 tablet_replica.cc:439] T 77fb0d40c7fd429398cdc88f6e6bb5bf P 3ce3f4e3b94e4538947cf5929cabdf88: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } } }
I20260807 11:23:06.383992 19578 tablet_replica.cc:442] T 77fb0d40c7fd429398cdc88f6e6bb5bf P 3ce3f4e3b94e4538947cf5929cabdf88: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 11:23:06.387807 19600 tablet_replica.cc:439] T 77fb0d40c7fd429398cdc88f6e6bb5bf P e191bf52edfd4d1ab10e61af6662ec82: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } } }
I20260807 11:23:06.388123 19600 tablet_replica.cc:442] T 77fb0d40c7fd429398cdc88f6e6bb5bf P e191bf52edfd4d1ab10e61af6662ec82: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 11:23:06.397323 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:06.390016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52546 (local address 127.16.197.1:46495)
0807 11:23:06.390206 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:06.390213 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:06.391006 (+ 793us) server_negotiation.cc:407] Connection header received
0807 11:23:06.391282 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:06.391294 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:06.391375 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:06.391538 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:23:06.392325 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:06.393176 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:06.394771 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:06.395469 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:06.395601 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:06.396037 (+ 436us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:23:06.396723 (+ 686us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:23:06.396884 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:06.396945 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 11:23:06.397034 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 11:23:06.414018 19567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:06.408393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:34007 (local address 127.16.197.2:38143)
0807 11:23:06.408616 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:06.408621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:06.409101 (+ 480us) server_negotiation.cc:407] Connection header received
0807 11:23:06.409324 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:06.409328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:06.409401 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:06.409501 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:23:06.410002 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:06.410882 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:06.412260 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:06.412686 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:06.412854 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:06.413277 (+ 423us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:23:06.413380 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:23:06.413480 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:06.413620 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 11:23:06.413704 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 11:23:06.414062 19619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:06.408259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:38143 (local address 127.16.197.1:34007)
0807 11:23:06.408992 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:06.409007 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:06.409173 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:06.409517 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:06.409522 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:06.409533 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 11:23:06.409808 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:06.409817 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:06.411058 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:06.411066 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:06.412111 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:06.412123 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:06.412877 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:06.412884 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:06.413017 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:06.413067 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 11:23:06.413481 (+ 414us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 11:23:06.413496 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 11:23:06.413595 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 11:23:06.413726 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":466,"threads_started":1}
I20260807 11:23:06.467248 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:06.448903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:52979 (local address 127.16.197.1:46495)
0807 11:23:06.449169 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:06.449175 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:06.460794 (+ 11619us) server_negotiation.cc:407] Connection header received
0807 11:23:06.461002 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:06.461008 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:06.461100 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:06.461344 (+ 244us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:23:06.462134 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:06.462927 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:06.465196 (+ 2269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:06.465826 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:06.465971 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:06.466536 (+ 565us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:23:06.466644 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:23:06.466714 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:06.466868 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 11:23:06.466935 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260807 11:23:06.467485 19621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:06.448447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:46495 (local address 127.16.197.2:52979)
0807 11:23:06.460650 (+ 12203us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:06.460668 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:06.460852 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:06.461537 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:06.461542 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:06.461555 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 11:23:06.461930 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:06.461944 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:06.463094 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:06.463100 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:06.465043 (+ 1943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:06.465060 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:06.466088 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:06.466096 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:06.466288 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:06.466367 (+ 79us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 11:23:06.466744 (+ 377us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 11:23:06.466763 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 11:23:06.466848 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 11:23:06.467036 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12010,"thread_start_us":12516,"threads_started":1}
I20260807 11:23:06.507216 19040 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:23:06.507535 19040 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:23:06.511729 19040 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7ec49b30eb064f35bb6da171d3cc2d99: Sending DeleteTablet for 3 replicas of tablet 1762723c861344e883dcafa9b772a4b2
I20260807 11:23:06.512919 19185 tablet_service.cc:1514] Processing DeleteTablet for tablet 1762723c861344e883dcafa9b772a4b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:06 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:52530
I20260807 11:23:06.513907 19332 tablet_service.cc:1514] Processing DeleteTablet for tablet 1762723c861344e883dcafa9b772a4b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:06 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:41058
I20260807 11:23:06.514545 19476 tablet_service.cc:1514] Processing DeleteTablet for tablet 1762723c861344e883dcafa9b772a4b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:06 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:53986
I20260807 11:23:06.515657 19040 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 11:23:06.516350 19040 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:23:06.526180 19184 tablet_service.cc:1467] Processing CreateTablet for tablet 11767f104aa64c2bb139479da49210d3 (DEFAULT_TABLE table=test-table [id=573109b837b245ad943975fee66da376]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:06.526548 19475 tablet_service.cc:1467] Processing CreateTablet for tablet 11767f104aa64c2bb139479da49210d3 (DEFAULT_TABLE table=test-table [id=573109b837b245ad943975fee66da376]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:06.526819 19184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11767f104aa64c2bb139479da49210d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:06.527196 19475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11767f104aa64c2bb139479da49210d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:06.531638 19330 tablet_service.cc:1467] Processing CreateTablet for tablet 11767f104aa64c2bb139479da49210d3 (DEFAULT_TABLE table=test-table [id=573109b837b245ad943975fee66da376]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:06.532235 19330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11767f104aa64c2bb139479da49210d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:06.539673 19626 tablet_replica.cc:330] T 1762723c861344e883dcafa9b772a4b2 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: stopping tablet replica
I20260807 11:23:06.540076 19625 tablet_replica.cc:330] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82: stopping tablet replica
I20260807 11:23:06.540638 19626 raft_consensus.cc:2229] T 1762723c861344e883dcafa9b772a4b2 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:06.541232 19625 raft_consensus.cc:2229] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82 [term 1 LEADER]: Raft consensus shutting down.
I20260807 11:23:06.541697 19626 raft_consensus.cc:2258] T 1762723c861344e883dcafa9b772a4b2 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:06.542368 19627 tablet_replica.cc:330] T 1762723c861344e883dcafa9b772a4b2 P 3ce3f4e3b94e4538947cf5929cabdf88: stopping tablet replica
I20260807 11:23:06.542766 19627 raft_consensus.cc:2229] T 1762723c861344e883dcafa9b772a4b2 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:06.544126 19627 raft_consensus.cc:2258] T 1762723c861344e883dcafa9b772a4b2 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:06.544423 19625 raft_consensus.cc:2258] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:06.545898 19627 ts_tablet_manager.cc:1905] T 1762723c861344e883dcafa9b772a4b2 P 3ce3f4e3b94e4538947cf5929cabdf88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:06.546682 19626 ts_tablet_manager.cc:1905] T 1762723c861344e883dcafa9b772a4b2 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:06.549809 19628 tablet_bootstrap.cc:492] T 11767f104aa64c2bb139479da49210d3 P 3ce3f4e3b94e4538947cf5929cabdf88: Bootstrap starting.
I20260807 11:23:06.550197 19625 ts_tablet_manager.cc:1905] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:06.551177 19570 tablet_bootstrap.cc:492] T 11767f104aa64c2bb139479da49210d3 P e191bf52edfd4d1ab10e61af6662ec82: Bootstrap starting.
I20260807 11:23:06.552031 19628 tablet_bootstrap.cc:654] T 11767f104aa64c2bb139479da49210d3 P 3ce3f4e3b94e4538947cf5929cabdf88: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:06.552955 19570 tablet_bootstrap.cc:654] T 11767f104aa64c2bb139479da49210d3 P e191bf52edfd4d1ab10e61af6662ec82: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:06.555886 19629 tablet_bootstrap.cc:492] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Bootstrap starting.
I20260807 11:23:06.556500 19627 ts_tablet_manager.cc:1918] T 1762723c861344e883dcafa9b772a4b2 P 3ce3f4e3b94e4538947cf5929cabdf88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 11:23:06.556671 19627 log.cc:1196] T 1762723c861344e883dcafa9b772a4b2 P 3ce3f4e3b94e4538947cf5929cabdf88: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-2/wal/wals/1762723c861344e883dcafa9b772a4b2
I20260807 11:23:06.557224 19627 ts_tablet_manager.cc:1939] T 1762723c861344e883dcafa9b772a4b2 P 3ce3f4e3b94e4538947cf5929cabdf88: Deleting consensus metadata
I20260807 11:23:06.557909 19629 tablet_bootstrap.cc:654] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:06.559167 19026 catalog_manager.cc:4841] TS 3ce3f4e3b94e4538947cf5929cabdf88 (127.16.197.3:34011): tablet 1762723c861344e883dcafa9b772a4b2 (table test-table [id=e369eabb47574107ae129e059d967458]) successfully deleted
I20260807 11:23:06.565045 19629 tablet_bootstrap.cc:492] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: No bootstrap required, opened a new log
I20260807 11:23:06.565258 19629 ts_tablet_manager.cc:1397] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260807 11:23:06.566174 19629 raft_consensus.cc:348] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:06.566519 19629 raft_consensus.cc:374] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:06.566627 19629 raft_consensus.cc:729] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7eeeb95e1d14ce8b75d19f9eb8b11fc, State: Initialized, Role: FOLLOWER
I20260807 11:23:06.566862 19629 consensus_queue.cc:260] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:06.568192 19629 ts_tablet_manager.cc:1428] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 11:23:06.582545 19626 ts_tablet_manager.cc:1918] T 1762723c861344e883dcafa9b772a4b2 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 11:23:06.585482 19570 tablet_bootstrap.cc:492] T 11767f104aa64c2bb139479da49210d3 P e191bf52edfd4d1ab10e61af6662ec82: No bootstrap required, opened a new log
I20260807 11:23:06.585831 19570 ts_tablet_manager.cc:1397] T 11767f104aa64c2bb139479da49210d3 P e191bf52edfd4d1ab10e61af6662ec82: Time spent bootstrapping tablet: real 0.035s user 0.003s sys 0.008s
I20260807 11:23:06.586983 19570 raft_consensus.cc:348] T 11767f104aa64c2bb139479da49210d3 P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:06.587574 19570 raft_consensus.cc:374] T 11767f104aa64c2bb139479da49210d3 P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:06.587743 19570 raft_consensus.cc:729] T 11767f104aa64c2bb139479da49210d3 P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e191bf52edfd4d1ab10e61af6662ec82, State: Initialized, Role: FOLLOWER
I20260807 11:23:06.588174 19570 consensus_queue.cc:260] T 11767f104aa64c2bb139479da49210d3 P e191bf52edfd4d1ab10e61af6662ec82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:06.592747 19570 ts_tablet_manager.cc:1428] T 11767f104aa64c2bb139479da49210d3 P e191bf52edfd4d1ab10e61af6662ec82: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260807 11:23:06.610332 19626 log.cc:1196] T 1762723c861344e883dcafa9b772a4b2 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-1/wal/wals/1762723c861344e883dcafa9b772a4b2
I20260807 11:23:06.611085 19626 ts_tablet_manager.cc:1939] T 1762723c861344e883dcafa9b772a4b2 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Deleting consensus metadata
I20260807 11:23:06.612848 19025 catalog_manager.cc:4841] TS c7eeeb95e1d14ce8b75d19f9eb8b11fc (127.16.197.2:38143): tablet 1762723c861344e883dcafa9b772a4b2 (table test-table [id=e369eabb47574107ae129e059d967458]) successfully deleted
I20260807 11:23:06.613433 19628 tablet_bootstrap.cc:492] T 11767f104aa64c2bb139479da49210d3 P 3ce3f4e3b94e4538947cf5929cabdf88: No bootstrap required, opened a new log
I20260807 11:23:06.613621 19628 ts_tablet_manager.cc:1397] T 11767f104aa64c2bb139479da49210d3 P 3ce3f4e3b94e4538947cf5929cabdf88: Time spent bootstrapping tablet: real 0.064s user 0.021s sys 0.029s
I20260807 11:23:06.614305 19628 raft_consensus.cc:348] T 11767f104aa64c2bb139479da49210d3 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:06.614526 19628 raft_consensus.cc:374] T 11767f104aa64c2bb139479da49210d3 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:06.614625 19628 raft_consensus.cc:729] T 11767f104aa64c2bb139479da49210d3 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce3f4e3b94e4538947cf5929cabdf88, State: Initialized, Role: FOLLOWER
I20260807 11:23:06.578899 19625 ts_tablet_manager.cc:1918] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 11:23:06.614920 19628 consensus_queue.cc:260] T 11767f104aa64c2bb139479da49210d3 P 3ce3f4e3b94e4538947cf5929cabdf88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:06.615552 19628 ts_tablet_manager.cc:1428] T 11767f104aa64c2bb139479da49210d3 P 3ce3f4e3b94e4538947cf5929cabdf88: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 11:23:06.615789 19625 log.cc:1196] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-0/wal/wals/1762723c861344e883dcafa9b772a4b2
I20260807 11:23:06.616887 19625 ts_tablet_manager.cc:1939] T 1762723c861344e883dcafa9b772a4b2 P e191bf52edfd4d1ab10e61af6662ec82: Deleting consensus metadata
I20260807 11:23:06.618826 19024 catalog_manager.cc:4841] TS e191bf52edfd4d1ab10e61af6662ec82 (127.16.197.1:46495): tablet 1762723c861344e883dcafa9b772a4b2 (table test-table [id=e369eabb47574107ae129e059d967458]) successfully deleted
I20260807 11:23:06.628206 19602 raft_consensus.cc:482] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:23:06.628402 19602 raft_consensus.cc:504] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:06.629081 19602 leader_election.cc:288] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e191bf52edfd4d1ab10e61af6662ec82 (127.16.197.1:46495), 3ce3f4e3b94e4538947cf5929cabdf88 (127.16.197.3:34011)
I20260807 11:23:06.629812 19498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11767f104aa64c2bb139479da49210d3" candidate_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" is_pre_election: true
I20260807 11:23:06.630033 19498 raft_consensus.cc:2454] T 11767f104aa64c2bb139479da49210d3 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7eeeb95e1d14ce8b75d19f9eb8b11fc in term 0.
I20260807 11:23:06.630798 19211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11767f104aa64c2bb139479da49210d3" candidate_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e191bf52edfd4d1ab10e61af6662ec82" is_pre_election: true
I20260807 11:23:06.631311 19283 leader_election.cc:302] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce3f4e3b94e4538947cf5929cabdf88, c7eeeb95e1d14ce8b75d19f9eb8b11fc; no voters:
I20260807 11:23:06.631639 19211 raft_consensus.cc:2454] T 11767f104aa64c2bb139479da49210d3 P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7eeeb95e1d14ce8b75d19f9eb8b11fc in term 0.
I20260807 11:23:06.632484 19602 raft_consensus.cc:2781] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:23:06.632608 19602 raft_consensus.cc:482] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:23:06.632689 19602 raft_consensus.cc:3037] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:06.636076 19602 raft_consensus.cc:504] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:06.637312 19211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11767f104aa64c2bb139479da49210d3" candidate_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e191bf52edfd4d1ab10e61af6662ec82"
I20260807 11:23:06.637526 19211 raft_consensus.cc:3037] T 11767f104aa64c2bb139479da49210d3 P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:06.640899 19211 raft_consensus.cc:2454] T 11767f104aa64c2bb139479da49210d3 P e191bf52edfd4d1ab10e61af6662ec82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7eeeb95e1d14ce8b75d19f9eb8b11fc in term 1.
I20260807 11:23:06.643013 19281 leader_election.cc:302] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7eeeb95e1d14ce8b75d19f9eb8b11fc, e191bf52edfd4d1ab10e61af6662ec82; no voters:
I20260807 11:23:06.643389 19575 raft_consensus.cc:2781] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:06.643541 19575 raft_consensus.cc:686] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 1 LEADER]: Becoming Leader. State: Replica: c7eeeb95e1d14ce8b75d19f9eb8b11fc, State: Running, Role: LEADER
I20260807 11:23:06.643814 19575 consensus_queue.cc:237] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:06.646888 19498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11767f104aa64c2bb139479da49210d3" candidate_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce3f4e3b94e4538947cf5929cabdf88"
I20260807 11:23:06.647298 19498 raft_consensus.cc:3037] T 11767f104aa64c2bb139479da49210d3 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:06.650560 19498 raft_consensus.cc:2454] T 11767f104aa64c2bb139479da49210d3 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7eeeb95e1d14ce8b75d19f9eb8b11fc in term 1.
I20260807 11:23:06.651548 19602 leader_election.cc:288] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [CANDIDATE]: Term 1 election: Requested vote from peers e191bf52edfd4d1ab10e61af6662ec82 (127.16.197.1:46495), 3ce3f4e3b94e4538947cf5929cabdf88 (127.16.197.3:34011)
I20260807 11:23:06.660892 19040 catalog_manager.cc:5495] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc reported cstate change: term changed from 0 to 1, leader changed from <none> to c7eeeb95e1d14ce8b75d19f9eb8b11fc (127.16.197.2). New cstate: current_term: 1 leader_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } health_report { overall_health: UNKNOWN } } }
I20260807 11:23:06.688371 19567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:06.681334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41078 (local address 127.16.197.2:38143)
0807 11:23:06.681564 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:06.681570 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:06.681587 (+ 17us) server_negotiation.cc:407] Connection header received
0807 11:23:06.681650 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:06.681655 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:06.681775 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:06.681863 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:23:06.682394 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:06.683585 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:06.685617 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:06.686335 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:06.686460 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:06.687084 (+ 624us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:23:06.687243 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:23:06.687324 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:06.687945 (+ 621us) server_negotiation.cc:299] Negotiation successful
0807 11:23:06.688026 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 11:23:06.704140 19621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:06.698695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:38143 (local address 127.16.197.2:36671)
0807 11:23:06.699020 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:06.699041 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:06.699195 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:06.699552 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:06.699556 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:06.699567 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 11:23:06.699834 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:06.699848 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:06.700887 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:06.700895 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:06.701967 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:06.701979 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:06.702893 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:06.702899 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:06.703096 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:06.703138 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 11:23:06.703559 (+ 421us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 11:23:06.703575 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 11:23:06.703645 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 11:23:06.703777 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":34}
I20260807 11:23:06.704144 19567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:06.698619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:36671 (local address 127.16.197.2:38143)
0807 11:23:06.698807 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:06.698813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:06.699159 (+ 346us) server_negotiation.cc:407] Connection header received
0807 11:23:06.699337 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:06.699341 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:06.699440 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:06.699540 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:23:06.699976 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:06.700755 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:06.702113 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:06.702634 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:06.702786 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:06.703325 (+ 539us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:23:06.703445 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:23:06.703549 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:06.703677 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 11:23:06.703753 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":57}
I20260807 11:23:06.714689 19498 raft_consensus.cc:1261] T 11767f104aa64c2bb139479da49210d3 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 1 FOLLOWER]: Refusing update from remote peer c7eeeb95e1d14ce8b75d19f9eb8b11fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:06.714857 19211 raft_consensus.cc:1261] T 11767f104aa64c2bb139479da49210d3 P e191bf52edfd4d1ab10e61af6662ec82 [term 1 FOLLOWER]: Refusing update from remote peer c7eeeb95e1d14ce8b75d19f9eb8b11fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:06.715411 19602 consensus_queue.cc:1035] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:06.715982 19575 consensus_queue.cc:1035] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 11:23:06.763206 19605 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 11:23:08.386552 19040 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:23:08.386849 19040 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:23:08.393116 19040 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7ec49b30eb064f35bb6da171d3cc2d99: Sending DeleteTablet for 3 replicas of tablet 11767f104aa64c2bb139479da49210d3
I20260807 11:23:08.394425 19184 tablet_service.cc:1514] Processing DeleteTablet for tablet 11767f104aa64c2bb139479da49210d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:08 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:52530
I20260807 11:23:08.394582 19330 tablet_service.cc:1514] Processing DeleteTablet for tablet 11767f104aa64c2bb139479da49210d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:08 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:41058
I20260807 11:23:08.394804 19475 tablet_service.cc:1514] Processing DeleteTablet for tablet 11767f104aa64c2bb139479da49210d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:08 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:53986
I20260807 11:23:08.406472 19660 tablet_replica.cc:330] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: stopping tablet replica
I20260807 11:23:08.407238 19660 raft_consensus.cc:2229] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 1 LEADER]: Raft consensus shutting down.
I20260807 11:23:08.407816 19661 tablet_replica.cc:330] T 11767f104aa64c2bb139479da49210d3 P 3ce3f4e3b94e4538947cf5929cabdf88: stopping tablet replica
I20260807 11:23:08.408258 19661 raft_consensus.cc:2229] T 11767f104aa64c2bb139479da49210d3 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:08.408711 19660 raft_consensus.cc:2258] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:08.408960 19661 raft_consensus.cc:2258] T 11767f104aa64c2bb139479da49210d3 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:08.410255 19659 tablet_replica.cc:330] T 11767f104aa64c2bb139479da49210d3 P e191bf52edfd4d1ab10e61af6662ec82: stopping tablet replica
I20260807 11:23:08.410605 19659 raft_consensus.cc:2229] T 11767f104aa64c2bb139479da49210d3 P e191bf52edfd4d1ab10e61af6662ec82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:08.410854 19661 ts_tablet_manager.cc:1905] T 11767f104aa64c2bb139479da49210d3 P 3ce3f4e3b94e4538947cf5929cabdf88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:08.411223 19659 raft_consensus.cc:2258] T 11767f104aa64c2bb139479da49210d3 P e191bf52edfd4d1ab10e61af6662ec82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:08.410938 19660 ts_tablet_manager.cc:1905] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:08.412348 19659 ts_tablet_manager.cc:1905] T 11767f104aa64c2bb139479da49210d3 P e191bf52edfd4d1ab10e61af6662ec82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:08.419649 19661 ts_tablet_manager.cc:1918] T 11767f104aa64c2bb139479da49210d3 P 3ce3f4e3b94e4538947cf5929cabdf88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 11:23:08.419976 19661 log.cc:1196] T 11767f104aa64c2bb139479da49210d3 P 3ce3f4e3b94e4538947cf5929cabdf88: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-2/wal/wals/11767f104aa64c2bb139479da49210d3
I20260807 11:23:08.420616 19661 ts_tablet_manager.cc:1939] T 11767f104aa64c2bb139479da49210d3 P 3ce3f4e3b94e4538947cf5929cabdf88: Deleting consensus metadata
I20260807 11:23:08.421582 19660 ts_tablet_manager.cc:1918] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 11:23:08.421928 19660 log.cc:1196] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-1/wal/wals/11767f104aa64c2bb139479da49210d3
I20260807 11:23:08.421908 19026 catalog_manager.cc:4841] TS 3ce3f4e3b94e4538947cf5929cabdf88 (127.16.197.3:34011): tablet 11767f104aa64c2bb139479da49210d3 (table test-table [id=573109b837b245ad943975fee66da376]) successfully deleted
I20260807 11:23:08.422410 19659 ts_tablet_manager.cc:1918] T 11767f104aa64c2bb139479da49210d3 P e191bf52edfd4d1ab10e61af6662ec82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 11:23:08.422704 19659 log.cc:1196] T 11767f104aa64c2bb139479da49210d3 P e191bf52edfd4d1ab10e61af6662ec82: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-0/wal/wals/11767f104aa64c2bb139479da49210d3
I20260807 11:23:08.422851 19660 ts_tablet_manager.cc:1939] T 11767f104aa64c2bb139479da49210d3 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Deleting consensus metadata
I20260807 11:23:08.423789 19659 ts_tablet_manager.cc:1939] T 11767f104aa64c2bb139479da49210d3 P e191bf52edfd4d1ab10e61af6662ec82: Deleting consensus metadata
I20260807 11:23:08.424571 19025 catalog_manager.cc:4841] TS c7eeeb95e1d14ce8b75d19f9eb8b11fc (127.16.197.2:38143): tablet 11767f104aa64c2bb139479da49210d3 (table test-table [id=573109b837b245ad943975fee66da376]) successfully deleted
I20260807 11:23:08.425086 19024 catalog_manager.cc:4841] TS e191bf52edfd4d1ab10e61af6662ec82 (127.16.197.1:46495): tablet 11767f104aa64c2bb139479da49210d3 (table test-table [id=573109b837b245ad943975fee66da376]) successfully deleted
Aug 07 11:23:08 dist-test-slave-2w3w krb5kdc[18992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101788, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 11:23:08 dist-test-slave-2w3w krb5kdc[18992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101788, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:08.477394 19669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:08.456449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34294 (local address 127.16.197.62:42523)
0807 11:23:08.457056 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:08.457061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:08.457154 (+ 93us) server_negotiation.cc:407] Connection header received
0807 11:23:08.457286 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:08.457290 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:08.457357 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:08.457461 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:08.458836 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:08.459705 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:08.461249 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:08.461654 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:08.461765 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:08.471180 (+ 9415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:08.471186 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:08.471190 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:08.471243 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:08.474669 (+ 3426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:08.475592 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:08.475596 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:08.475599 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:08.475749 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:08.476177 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:08.476181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:08.476182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:08.476474 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:08.476667 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:08.476918 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 11:23:08.477099 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":325,"threads_started":1}
I20260807 11:23:08.481117 19040 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34294:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 11:23:08.481830 19040 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:23:08.488718 19330 tablet_service.cc:1467] Processing CreateTablet for tablet 2a5fa8feeb0d4c30a7500932350beb98 (DEFAULT_TABLE table=test-table [id=1c7e6f75219d41e4807a66a9c91c14ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:08.488760 19184 tablet_service.cc:1467] Processing CreateTablet for tablet 2a5fa8feeb0d4c30a7500932350beb98 (DEFAULT_TABLE table=test-table [id=1c7e6f75219d41e4807a66a9c91c14ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:08.488862 19475 tablet_service.cc:1467] Processing CreateTablet for tablet 2a5fa8feeb0d4c30a7500932350beb98 (DEFAULT_TABLE table=test-table [id=1c7e6f75219d41e4807a66a9c91c14ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:08.489293 19184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a5fa8feeb0d4c30a7500932350beb98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:08.489336 19330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a5fa8feeb0d4c30a7500932350beb98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:08.489318 19475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a5fa8feeb0d4c30a7500932350beb98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:08.500289 19671 tablet_bootstrap.cc:492] T 2a5fa8feeb0d4c30a7500932350beb98 P 3ce3f4e3b94e4538947cf5929cabdf88: Bootstrap starting.
I20260807 11:23:08.501135 19670 tablet_bootstrap.cc:492] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82: Bootstrap starting.
I20260807 11:23:08.503105 19672 tablet_bootstrap.cc:492] T 2a5fa8feeb0d4c30a7500932350beb98 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Bootstrap starting.
I20260807 11:23:08.505227 19670 tablet_bootstrap.cc:654] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:08.505695 19671 tablet_bootstrap.cc:654] T 2a5fa8feeb0d4c30a7500932350beb98 P 3ce3f4e3b94e4538947cf5929cabdf88: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:08.505954 19672 tablet_bootstrap.cc:654] T 2a5fa8feeb0d4c30a7500932350beb98 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:08.511802 19670 tablet_bootstrap.cc:492] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82: No bootstrap required, opened a new log
I20260807 11:23:08.512006 19671 tablet_bootstrap.cc:492] T 2a5fa8feeb0d4c30a7500932350beb98 P 3ce3f4e3b94e4538947cf5929cabdf88: No bootstrap required, opened a new log
I20260807 11:23:08.512079 19670 ts_tablet_manager.cc:1397] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.006s
I20260807 11:23:08.512248 19671 ts_tablet_manager.cc:1397] T 2a5fa8feeb0d4c30a7500932350beb98 P 3ce3f4e3b94e4538947cf5929cabdf88: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.009s
I20260807 11:23:08.512647 19672 tablet_bootstrap.cc:492] T 2a5fa8feeb0d4c30a7500932350beb98 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: No bootstrap required, opened a new log
I20260807 11:23:08.512871 19672 ts_tablet_manager.cc:1397] T 2a5fa8feeb0d4c30a7500932350beb98 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260807 11:23:08.513674 19670 raft_consensus.cc:348] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:08.514011 19672 raft_consensus.cc:348] T 2a5fa8feeb0d4c30a7500932350beb98 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:08.514159 19670 raft_consensus.cc:374] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:08.514382 19672 raft_consensus.cc:374] T 2a5fa8feeb0d4c30a7500932350beb98 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:08.514480 19670 raft_consensus.cc:729] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e191bf52edfd4d1ab10e61af6662ec82, State: Initialized, Role: FOLLOWER
I20260807 11:23:08.514624 19672 raft_consensus.cc:729] T 2a5fa8feeb0d4c30a7500932350beb98 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7eeeb95e1d14ce8b75d19f9eb8b11fc, State: Initialized, Role: FOLLOWER
I20260807 11:23:08.514936 19670 consensus_queue.cc:260] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:08.515120 19672 consensus_queue.cc:260] T 2a5fa8feeb0d4c30a7500932350beb98 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:08.515872 19672 ts_tablet_manager.cc:1428] T 2a5fa8feeb0d4c30a7500932350beb98 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 11:23:08.515939 19671 raft_consensus.cc:348] T 2a5fa8feeb0d4c30a7500932350beb98 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:08.516424 19671 raft_consensus.cc:374] T 2a5fa8feeb0d4c30a7500932350beb98 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:08.516662 19671 raft_consensus.cc:729] T 2a5fa8feeb0d4c30a7500932350beb98 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce3f4e3b94e4538947cf5929cabdf88, State: Initialized, Role: FOLLOWER
I20260807 11:23:08.517144 19671 consensus_queue.cc:260] T 2a5fa8feeb0d4c30a7500932350beb98 P 3ce3f4e3b94e4538947cf5929cabdf88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:08.517637 19670 ts_tablet_manager.cc:1428] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260807 11:23:08.518841 19671 ts_tablet_manager.cc:1428] T 2a5fa8feeb0d4c30a7500932350beb98 P 3ce3f4e3b94e4538947cf5929cabdf88: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260807 11:23:08.664373 19676 raft_consensus.cc:482] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:23:08.664757 19676 raft_consensus.cc:504] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:08.666214 19676 leader_election.cc:288] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7eeeb95e1d14ce8b75d19f9eb8b11fc (127.16.197.2:38143), 3ce3f4e3b94e4538947cf5929cabdf88 (127.16.197.3:34011)
I20260807 11:23:08.666826 19497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a5fa8feeb0d4c30a7500932350beb98" candidate_uuid: "e191bf52edfd4d1ab10e61af6662ec82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" is_pre_election: true
I20260807 11:23:08.666934 19358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a5fa8feeb0d4c30a7500932350beb98" candidate_uuid: "e191bf52edfd4d1ab10e61af6662ec82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" is_pre_election: true
I20260807 11:23:08.667275 19497 raft_consensus.cc:2454] T 2a5fa8feeb0d4c30a7500932350beb98 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e191bf52edfd4d1ab10e61af6662ec82 in term 0.
I20260807 11:23:08.667346 19358 raft_consensus.cc:2454] T 2a5fa8feeb0d4c30a7500932350beb98 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e191bf52edfd4d1ab10e61af6662ec82 in term 0.
I20260807 11:23:08.667968 19135 leader_election.cc:302] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce3f4e3b94e4538947cf5929cabdf88, e191bf52edfd4d1ab10e61af6662ec82; no voters:
I20260807 11:23:08.668355 19676 raft_consensus.cc:2781] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:23:08.668576 19676 raft_consensus.cc:482] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:23:08.668746 19676 raft_consensus.cc:3037] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:08.671666 19676 raft_consensus.cc:504] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:08.672590 19676 leader_election.cc:288] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82 [CANDIDATE]: Term 1 election: Requested vote from peers c7eeeb95e1d14ce8b75d19f9eb8b11fc (127.16.197.2:38143), 3ce3f4e3b94e4538947cf5929cabdf88 (127.16.197.3:34011)
I20260807 11:23:08.673053 19358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a5fa8feeb0d4c30a7500932350beb98" candidate_uuid: "e191bf52edfd4d1ab10e61af6662ec82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc"
I20260807 11:23:08.673216 19497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a5fa8feeb0d4c30a7500932350beb98" candidate_uuid: "e191bf52edfd4d1ab10e61af6662ec82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce3f4e3b94e4538947cf5929cabdf88"
I20260807 11:23:08.673448 19358 raft_consensus.cc:3037] T 2a5fa8feeb0d4c30a7500932350beb98 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:08.673588 19497 raft_consensus.cc:3037] T 2a5fa8feeb0d4c30a7500932350beb98 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:08.678045 19497 raft_consensus.cc:2454] T 2a5fa8feeb0d4c30a7500932350beb98 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e191bf52edfd4d1ab10e61af6662ec82 in term 1.
I20260807 11:23:08.678045 19358 raft_consensus.cc:2454] T 2a5fa8feeb0d4c30a7500932350beb98 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e191bf52edfd4d1ab10e61af6662ec82 in term 1.
I20260807 11:23:08.678738 19135 leader_election.cc:302] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce3f4e3b94e4538947cf5929cabdf88, e191bf52edfd4d1ab10e61af6662ec82; no voters:
I20260807 11:23:08.679219 19676 raft_consensus.cc:2781] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:08.680037 19676 raft_consensus.cc:686] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82 [term 1 LEADER]: Becoming Leader. State: Replica: e191bf52edfd4d1ab10e61af6662ec82, State: Running, Role: LEADER
I20260807 11:23:08.680481 19676 consensus_queue.cc:237] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:08.684099 19039 catalog_manager.cc:5495] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82 reported cstate change: term changed from 0 to 1, leader changed from <none> to e191bf52edfd4d1ab10e61af6662ec82 (127.16.197.1). New cstate: current_term: 1 leader_uuid: "e191bf52edfd4d1ab10e61af6662ec82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } health_report { overall_health: UNKNOWN } } }
I20260807 11:23:08.715917 19681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:08.706210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40822 (local address 127.16.197.1:46495)
0807 11:23:08.707536 (+ 1326us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:08.707540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:08.707614 (+ 74us) server_negotiation.cc:407] Connection header received
0807 11:23:08.707671 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:08.707674 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:08.707748 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:08.707832 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:23:08.708750 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:08.709959 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:08.713219 (+ 3260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:08.713986 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:08.714091 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:08.714838 (+ 747us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:23:08.714948 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:23:08.715078 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:08.715241 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 11:23:08.715311 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1138,"thread_start_us":923,"threads_started":1}
I20260807 11:23:08.728175 19358 raft_consensus.cc:1261] T 2a5fa8feeb0d4c30a7500932350beb98 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 1 FOLLOWER]: Refusing update from remote peer e191bf52edfd4d1ab10e61af6662ec82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:08.728175 19497 raft_consensus.cc:1261] T 2a5fa8feeb0d4c30a7500932350beb98 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 1 FOLLOWER]: Refusing update from remote peer e191bf52edfd4d1ab10e61af6662ec82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:08.728888 19678 consensus_queue.cc:1035] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 11:23:08.729542 19676 consensus_queue.cc:1035] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:08.792922 19036 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:23:08.793216 19036 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:23:08.796967 19036 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7ec49b30eb064f35bb6da171d3cc2d99: Sending DeleteTablet for 3 replicas of tablet 2a5fa8feeb0d4c30a7500932350beb98
I20260807 11:23:08.798106 19475 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a5fa8feeb0d4c30a7500932350beb98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:08 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:53986
I20260807 11:23:08.798143 19184 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a5fa8feeb0d4c30a7500932350beb98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:08 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:52530
I20260807 11:23:08.798143 19330 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a5fa8feeb0d4c30a7500932350beb98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:08 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:41058
I20260807 11:23:08.798514 19661 tablet_replica.cc:330] T 2a5fa8feeb0d4c30a7500932350beb98 P 3ce3f4e3b94e4538947cf5929cabdf88: stopping tablet replica
I20260807 11:23:08.798573 19659 tablet_replica.cc:330] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82: stopping tablet replica
I20260807 11:23:08.798862 19661 raft_consensus.cc:2229] T 2a5fa8feeb0d4c30a7500932350beb98 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:08.798935 19659 raft_consensus.cc:2229] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82 [term 1 LEADER]: Raft consensus shutting down.
I20260807 11:23:08.799201 19661 raft_consensus.cc:2258] T 2a5fa8feeb0d4c30a7500932350beb98 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:08.799422 19659 raft_consensus.cc:2258] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:08.799635 19660 tablet_replica.cc:330] T 2a5fa8feeb0d4c30a7500932350beb98 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: stopping tablet replica
I20260807 11:23:08.800040 19660 raft_consensus.cc:2229] T 2a5fa8feeb0d4c30a7500932350beb98 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:08.799693 19036 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34294:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 11:23:08.800336 19660 raft_consensus.cc:2258] T 2a5fa8feeb0d4c30a7500932350beb98 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:08.800603 19659 ts_tablet_manager.cc:1905] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 11:23:08.800920 19036 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:23:08.801204 19661 ts_tablet_manager.cc:1905] T 2a5fa8feeb0d4c30a7500932350beb98 P 3ce3f4e3b94e4538947cf5929cabdf88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:08.801534 19660 ts_tablet_manager.cc:1905] T 2a5fa8feeb0d4c30a7500932350beb98 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:08.809772 19659 ts_tablet_manager.cc:1918] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 11:23:08.810089 19659 log.cc:1196] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-0/wal/wals/2a5fa8feeb0d4c30a7500932350beb98
I20260807 11:23:08.810858 19659 ts_tablet_manager.cc:1939] T 2a5fa8feeb0d4c30a7500932350beb98 P e191bf52edfd4d1ab10e61af6662ec82: Deleting consensus metadata
I20260807 11:23:08.811168 19330 tablet_service.cc:1467] Processing CreateTablet for tablet 151b3254bfa24e1d80bff13426406da5 (DEFAULT_TABLE table=test-table [id=e5eaf1fca5c643bc80bf9ffeef4bfc95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:08.811532 19660 ts_tablet_manager.cc:1918] T 2a5fa8feeb0d4c30a7500932350beb98 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 11:23:08.811626 19330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 151b3254bfa24e1d80bff13426406da5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:08.811676 19184 tablet_service.cc:1467] Processing CreateTablet for tablet 151b3254bfa24e1d80bff13426406da5 (DEFAULT_TABLE table=test-table [id=e5eaf1fca5c643bc80bf9ffeef4bfc95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:08.811836 19660 log.cc:1196] T 2a5fa8feeb0d4c30a7500932350beb98 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-1/wal/wals/2a5fa8feeb0d4c30a7500932350beb98
I20260807 11:23:08.812225 19184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 151b3254bfa24e1d80bff13426406da5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:08.812515 19660 ts_tablet_manager.cc:1939] T 2a5fa8feeb0d4c30a7500932350beb98 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Deleting consensus metadata
I20260807 11:23:08.813705 19025 catalog_manager.cc:4841] TS c7eeeb95e1d14ce8b75d19f9eb8b11fc (127.16.197.2:38143): tablet 2a5fa8feeb0d4c30a7500932350beb98 (table test-table [id=1c7e6f75219d41e4807a66a9c91c14ae]) successfully deleted
I20260807 11:23:08.815315 19024 catalog_manager.cc:4841] TS e191bf52edfd4d1ab10e61af6662ec82 (127.16.197.1:46495): tablet 2a5fa8feeb0d4c30a7500932350beb98 (table test-table [id=1c7e6f75219d41e4807a66a9c91c14ae]) successfully deleted
I20260807 11:23:08.809772 19661 ts_tablet_manager.cc:1918] T 2a5fa8feeb0d4c30a7500932350beb98 P 3ce3f4e3b94e4538947cf5929cabdf88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 11:23:08.810827 19475 tablet_service.cc:1467] Processing CreateTablet for tablet 151b3254bfa24e1d80bff13426406da5 (DEFAULT_TABLE table=test-table [id=e5eaf1fca5c643bc80bf9ffeef4bfc95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:08.819561 19475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 151b3254bfa24e1d80bff13426406da5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:08.821416 19670 tablet_bootstrap.cc:492] T 151b3254bfa24e1d80bff13426406da5 P e191bf52edfd4d1ab10e61af6662ec82: Bootstrap starting.
I20260807 11:23:08.824391 19670 tablet_bootstrap.cc:654] T 151b3254bfa24e1d80bff13426406da5 P e191bf52edfd4d1ab10e61af6662ec82: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:08.824695 19661 log.cc:1196] T 2a5fa8feeb0d4c30a7500932350beb98 P 3ce3f4e3b94e4538947cf5929cabdf88: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-2/wal/wals/2a5fa8feeb0d4c30a7500932350beb98
I20260807 11:23:08.825049 19672 tablet_bootstrap.cc:492] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Bootstrap starting.
I20260807 11:23:08.826656 19671 tablet_bootstrap.cc:492] T 151b3254bfa24e1d80bff13426406da5 P 3ce3f4e3b94e4538947cf5929cabdf88: Bootstrap starting.
I20260807 11:23:08.827109 19661 ts_tablet_manager.cc:1939] T 2a5fa8feeb0d4c30a7500932350beb98 P 3ce3f4e3b94e4538947cf5929cabdf88: Deleting consensus metadata
I20260807 11:23:08.828861 19672 tablet_bootstrap.cc:654] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:08.829200 19026 catalog_manager.cc:4841] TS 3ce3f4e3b94e4538947cf5929cabdf88 (127.16.197.3:34011): tablet 2a5fa8feeb0d4c30a7500932350beb98 (table test-table [id=1c7e6f75219d41e4807a66a9c91c14ae]) successfully deleted
I20260807 11:23:08.830222 19671 tablet_bootstrap.cc:654] T 151b3254bfa24e1d80bff13426406da5 P 3ce3f4e3b94e4538947cf5929cabdf88: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:08.831573 19670 tablet_bootstrap.cc:492] T 151b3254bfa24e1d80bff13426406da5 P e191bf52edfd4d1ab10e61af6662ec82: No bootstrap required, opened a new log
I20260807 11:23:08.832130 19670 ts_tablet_manager.cc:1397] T 151b3254bfa24e1d80bff13426406da5 P e191bf52edfd4d1ab10e61af6662ec82: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260807 11:23:08.834019 19670 raft_consensus.cc:348] T 151b3254bfa24e1d80bff13426406da5 P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:08.834293 19672 tablet_bootstrap.cc:492] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: No bootstrap required, opened a new log
I20260807 11:23:08.834522 19670 raft_consensus.cc:374] T 151b3254bfa24e1d80bff13426406da5 P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:08.834628 19672 ts_tablet_manager.cc:1397] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260807 11:23:08.834745 19670 raft_consensus.cc:729] T 151b3254bfa24e1d80bff13426406da5 P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e191bf52edfd4d1ab10e61af6662ec82, State: Initialized, Role: FOLLOWER
I20260807 11:23:08.835284 19671 tablet_bootstrap.cc:492] T 151b3254bfa24e1d80bff13426406da5 P 3ce3f4e3b94e4538947cf5929cabdf88: No bootstrap required, opened a new log
I20260807 11:23:08.835147 19670 consensus_queue.cc:260] T 151b3254bfa24e1d80bff13426406da5 P e191bf52edfd4d1ab10e61af6662ec82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:08.835681 19671 ts_tablet_manager.cc:1397] T 151b3254bfa24e1d80bff13426406da5 P 3ce3f4e3b94e4538947cf5929cabdf88: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260807 11:23:08.836462 19672 raft_consensus.cc:348] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:08.836752 19670 ts_tablet_manager.cc:1428] T 151b3254bfa24e1d80bff13426406da5 P e191bf52edfd4d1ab10e61af6662ec82: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260807 11:23:08.836864 19672 raft_consensus.cc:374] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:08.837121 19672 raft_consensus.cc:729] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7eeeb95e1d14ce8b75d19f9eb8b11fc, State: Initialized, Role: FOLLOWER
I20260807 11:23:08.837146 19671 raft_consensus.cc:348] T 151b3254bfa24e1d80bff13426406da5 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:08.837584 19671 raft_consensus.cc:374] T 151b3254bfa24e1d80bff13426406da5 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:08.837767 19671 raft_consensus.cc:729] T 151b3254bfa24e1d80bff13426406da5 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce3f4e3b94e4538947cf5929cabdf88, State: Initialized, Role: FOLLOWER
I20260807 11:23:08.837584 19672 consensus_queue.cc:260] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:08.838182 19671 consensus_queue.cc:260] T 151b3254bfa24e1d80bff13426406da5 P 3ce3f4e3b94e4538947cf5929cabdf88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:08.839156 19671 ts_tablet_manager.cc:1428] T 151b3254bfa24e1d80bff13426406da5 P 3ce3f4e3b94e4538947cf5929cabdf88: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 11:23:08.843904 19672 ts_tablet_manager.cc:1428] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20260807 11:23:08.929073 19641 raft_consensus.cc:482] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:23:08.929543 19641 raft_consensus.cc:504] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:08.930806 19641 leader_election.cc:288] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e191bf52edfd4d1ab10e61af6662ec82 (127.16.197.1:46495), 3ce3f4e3b94e4538947cf5929cabdf88 (127.16.197.3:34011)
I20260807 11:23:08.931596 19497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "151b3254bfa24e1d80bff13426406da5" candidate_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" is_pre_election: true
I20260807 11:23:08.931697 19211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "151b3254bfa24e1d80bff13426406da5" candidate_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e191bf52edfd4d1ab10e61af6662ec82" is_pre_election: true
I20260807 11:23:08.932049 19497 raft_consensus.cc:2454] T 151b3254bfa24e1d80bff13426406da5 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7eeeb95e1d14ce8b75d19f9eb8b11fc in term 0.
I20260807 11:23:08.932113 19211 raft_consensus.cc:2454] T 151b3254bfa24e1d80bff13426406da5 P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7eeeb95e1d14ce8b75d19f9eb8b11fc in term 0.
I20260807 11:23:08.932925 19281 leader_election.cc:302] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3ce3f4e3b94e4538947cf5929cabdf88, c7eeeb95e1d14ce8b75d19f9eb8b11fc, e191bf52edfd4d1ab10e61af6662ec82; no voters:
I20260807 11:23:08.933472 19641 raft_consensus.cc:2781] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:23:08.933867 19641 raft_consensus.cc:482] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:23:08.934070 19641 raft_consensus.cc:3037] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:08.937536 19641 raft_consensus.cc:504] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:08.938391 19641 leader_election.cc:288] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [CANDIDATE]: Term 1 election: Requested vote from peers e191bf52edfd4d1ab10e61af6662ec82 (127.16.197.1:46495), 3ce3f4e3b94e4538947cf5929cabdf88 (127.16.197.3:34011)
I20260807 11:23:08.939126 19497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "151b3254bfa24e1d80bff13426406da5" candidate_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce3f4e3b94e4538947cf5929cabdf88"
I20260807 11:23:08.939188 19211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "151b3254bfa24e1d80bff13426406da5" candidate_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e191bf52edfd4d1ab10e61af6662ec82"
I20260807 11:23:08.939589 19211 raft_consensus.cc:3037] T 151b3254bfa24e1d80bff13426406da5 P e191bf52edfd4d1ab10e61af6662ec82 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:08.939615 19497 raft_consensus.cc:3037] T 151b3254bfa24e1d80bff13426406da5 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:08.944756 19497 raft_consensus.cc:2454] T 151b3254bfa24e1d80bff13426406da5 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7eeeb95e1d14ce8b75d19f9eb8b11fc in term 1.
I20260807 11:23:08.944818 19211 raft_consensus.cc:2454] T 151b3254bfa24e1d80bff13426406da5 P e191bf52edfd4d1ab10e61af6662ec82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7eeeb95e1d14ce8b75d19f9eb8b11fc in term 1.
I20260807 11:23:08.945544 19283 leader_election.cc:302] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce3f4e3b94e4538947cf5929cabdf88, c7eeeb95e1d14ce8b75d19f9eb8b11fc; no voters:
I20260807 11:23:08.946169 19641 raft_consensus.cc:2781] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:08.946486 19641 raft_consensus.cc:686] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 1 LEADER]: Becoming Leader. State: Replica: c7eeeb95e1d14ce8b75d19f9eb8b11fc, State: Running, Role: LEADER
I20260807 11:23:08.946913 19641 consensus_queue.cc:237] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } }
I20260807 11:23:08.951120 19039 catalog_manager.cc:5495] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc reported cstate change: term changed from 0 to 1, leader changed from <none> to c7eeeb95e1d14ce8b75d19f9eb8b11fc (127.16.197.2). New cstate: current_term: 1 leader_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7eeeb95e1d14ce8b75d19f9eb8b11fc" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 } health_report { overall_health: UNKNOWN } } }
I20260807 11:23:09.001513 19691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:08.993407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49444 (local address 127.16.197.2:38143)
0807 11:23:08.994682 (+ 1275us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:08.994686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:08.994755 (+ 69us) server_negotiation.cc:407] Connection header received
0807 11:23:08.994808 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:08.994812 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:08.994900 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:08.995071 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:23:08.995794 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:08.997015 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:08.998707 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:08.999518 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:08.999665 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:09.000388 (+ 723us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:23:09.000526 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:23:09.000636 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:09.000794 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 11:23:09.000878 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1090,"thread_start_us":805,"threads_started":1}
I20260807 11:23:09.012646 19497 raft_consensus.cc:1261] T 151b3254bfa24e1d80bff13426406da5 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 1 FOLLOWER]: Refusing update from remote peer c7eeeb95e1d14ce8b75d19f9eb8b11fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:09.012727 19211 raft_consensus.cc:1261] T 151b3254bfa24e1d80bff13426406da5 P e191bf52edfd4d1ab10e61af6662ec82 [term 1 FOLLOWER]: Refusing update from remote peer c7eeeb95e1d14ce8b75d19f9eb8b11fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:09.013319 19641 consensus_queue.cc:1035] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce3f4e3b94e4538947cf5929cabdf88" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 34011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:09.013818 19647 consensus_queue.cc:1035] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e191bf52edfd4d1ab10e61af6662ec82" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 46495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 11:23:09.041987 19639 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 11:23:10.657055 19039 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:23:10.657475 19039 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:23:10.663245 19039 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7ec49b30eb064f35bb6da171d3cc2d99: Sending DeleteTablet for 3 replicas of tablet 151b3254bfa24e1d80bff13426406da5
I20260807 11:23:10.664326 19184 tablet_service.cc:1514] Processing DeleteTablet for tablet 151b3254bfa24e1d80bff13426406da5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:10 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:52530
I20260807 11:23:10.664698 19475 tablet_service.cc:1514] Processing DeleteTablet for tablet 151b3254bfa24e1d80bff13426406da5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:10 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:53986
I20260807 11:23:10.665411 19332 tablet_service.cc:1514] Processing DeleteTablet for tablet 151b3254bfa24e1d80bff13426406da5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:10 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:41058
I20260807 11:23:10.665516 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 19115
I20260807 11:23:10.676095 19719 tablet_replica.cc:330] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: stopping tablet replica
I20260807 11:23:10.677495 19719 raft_consensus.cc:2229] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 1 LEADER]: Raft consensus shutting down.
I20260807 11:23:10.678757 19719 raft_consensus.cc:2258] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:10.680864 19719 ts_tablet_manager.cc:1905] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:10.683583 19717 tablet_replica.cc:330] T 151b3254bfa24e1d80bff13426406da5 P e191bf52edfd4d1ab10e61af6662ec82: stopping tablet replica
I20260807 11:23:10.683943 19717 raft_consensus.cc:2229] T 151b3254bfa24e1d80bff13426406da5 P e191bf52edfd4d1ab10e61af6662ec82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:10.684340 19717 raft_consensus.cc:2258] T 151b3254bfa24e1d80bff13426406da5 P e191bf52edfd4d1ab10e61af6662ec82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:10.685067 19718 tablet_replica.cc:330] T 151b3254bfa24e1d80bff13426406da5 P 3ce3f4e3b94e4538947cf5929cabdf88: stopping tablet replica
I20260807 11:23:10.685339 19718 raft_consensus.cc:2229] T 151b3254bfa24e1d80bff13426406da5 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:10.685663 19718 raft_consensus.cc:2258] T 151b3254bfa24e1d80bff13426406da5 P 3ce3f4e3b94e4538947cf5929cabdf88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:10.685711 19717 ts_tablet_manager.cc:1905] T 151b3254bfa24e1d80bff13426406da5 P e191bf52edfd4d1ab10e61af6662ec82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:10.688995 19718 ts_tablet_manager.cc:1905] T 151b3254bfa24e1d80bff13426406da5 P 3ce3f4e3b94e4538947cf5929cabdf88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:10.691856 19719 ts_tablet_manager.cc:1918] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 11:23:10.692088 19719 log.cc:1196] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-1/wal/wals/151b3254bfa24e1d80bff13426406da5
I20260807 11:23:10.692843 19719 ts_tablet_manager.cc:1939] T 151b3254bfa24e1d80bff13426406da5 P c7eeeb95e1d14ce8b75d19f9eb8b11fc: Deleting consensus metadata
I20260807 11:23:10.694370 19025 catalog_manager.cc:4841] TS c7eeeb95e1d14ce8b75d19f9eb8b11fc (127.16.197.2:38143): tablet 151b3254bfa24e1d80bff13426406da5 (table test-table [id=e5eaf1fca5c643bc80bf9ffeef4bfc95]) successfully deleted
I20260807 11:23:10.697487 19717 ts_tablet_manager.cc:1918] T 151b3254bfa24e1d80bff13426406da5 P e191bf52edfd4d1ab10e61af6662ec82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 11:23:10.697688 19717 log.cc:1196] T 151b3254bfa24e1d80bff13426406da5 P e191bf52edfd4d1ab10e61af6662ec82: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-0/wal/wals/151b3254bfa24e1d80bff13426406da5
I20260807 11:23:10.698649 19717 ts_tablet_manager.cc:1939] T 151b3254bfa24e1d80bff13426406da5 P e191bf52edfd4d1ab10e61af6662ec82: Deleting consensus metadata
I20260807 11:23:10.698684 19718 ts_tablet_manager.cc:1918] T 151b3254bfa24e1d80bff13426406da5 P 3ce3f4e3b94e4538947cf5929cabdf88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 11:23:10.698935 19718 log.cc:1196] T 151b3254bfa24e1d80bff13426406da5 P 3ce3f4e3b94e4538947cf5929cabdf88: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786101765888975-17172-0/minicluster-data/ts-2/wal/wals/151b3254bfa24e1d80bff13426406da5
I20260807 11:23:10.699666 19718 ts_tablet_manager.cc:1939] T 151b3254bfa24e1d80bff13426406da5 P 3ce3f4e3b94e4538947cf5929cabdf88: Deleting consensus metadata
I20260807 11:23:10.700413 19024 catalog_manager.cc:4841] TS e191bf52edfd4d1ab10e61af6662ec82 (127.16.197.1:46495): tablet 151b3254bfa24e1d80bff13426406da5 (table test-table [id=e5eaf1fca5c643bc80bf9ffeef4bfc95]) successfully deleted
I20260807 11:23:10.700804 19026 catalog_manager.cc:4841] TS 3ce3f4e3b94e4538947cf5929cabdf88 (127.16.197.3:34011): tablet 151b3254bfa24e1d80bff13426406da5 (table test-table [id=e5eaf1fca5c643bc80bf9ffeef4bfc95]) successfully deleted
Aug 07 11:23:10 dist-test-slave-2w3w krb5kdc[18992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101788, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.197.1@KRBTEST.COM
I20260807 11:23:10.712100 19720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:10.667219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40830 (local address 127.16.197.1:46495)
0807 11:23:10.685465 (+ 18246us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:10.685472 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:10.685501 (+ 29us) server_negotiation.cc:407] Connection header received
0807 11:23:10.685568 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:10.685573 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:10.685664 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:10.688417 (+ 2753us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:10.688733 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:10.690558 (+ 1825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:10.694924 (+ 4366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:10.696047 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:10.696248 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:10.704237 (+ 7989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:10.704257 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:10.704275 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:10.704312 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:10.708520 (+ 4208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:10.709420 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:10.709427 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:10.709429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:10.709624 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:10.710226 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:10.710230 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:10.710232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:10.710706 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:10.711234 (+ 528us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:10.711397 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 11:23:10.711639 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18061,"thread_start_us":21909,"threads_started":1}
W20260807 11:23:10.713014 19254 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40830
W20260807 11:23:10.713846 17172 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 19115: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 11:23:10.714203 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 19115
I20260807 11:23:10.739512 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 19264
Aug 07 11:23:10 dist-test-slave-2w3w krb5kdc[18992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101788, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.197.2@KRBTEST.COM
I20260807 11:23:10.756938 19722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:10.740510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49456 (local address 127.16.197.2:38143)
0807 11:23:10.741126 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:10.741130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:10.741146 (+ 16us) server_negotiation.cc:407] Connection header received
0807 11:23:10.741200 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:10.741203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:10.741306 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:10.741427 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:10.743513 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:10.744293 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:10.745959 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:10.746388 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:10.746529 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:10.751378 (+ 4849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:10.751394 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:10.751410 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:10.751447 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:10.754464 (+ 3017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:10.755179 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:10.755187 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:10.755189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:10.755332 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:10.755662 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:10.755665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:10.755667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:10.756032 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:10.756289 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:10.756473 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 11:23:10.756649 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":343,"threads_started":1}
W20260807 11:23:10.757719 19399 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49456
W20260807 11:23:10.758462 17172 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 19264: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 11:23:10.758626 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 19264
I20260807 11:23:10.785305 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 19408
Aug 07 11:23:10 dist-test-slave-2w3w krb5kdc[18992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101788, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.197.3@KRBTEST.COM
I20260807 11:23:10.800997 19724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:10.786078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52744 (local address 127.16.197.3:34011)
0807 11:23:10.786632 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:10.786637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:10.786652 (+ 15us) server_negotiation.cc:407] Connection header received
0807 11:23:10.786707 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:10.786711 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:10.786771 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:10.786901 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:10.788291 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:10.789054 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:10.790344 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:10.790765 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:10.790906 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:10.795334 (+ 4428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:10.795348 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:10.795365 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:10.795403 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:10.798128 (+ 2725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:10.798809 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:10.798816 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:10.798818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:10.798952 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:10.799384 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:10.799390 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:10.799393 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:10.799747 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:10.800057 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:10.800430 (+ 373us) server_negotiation.cc:299] Negotiation successful
0807 11:23:10.800677 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":244,"threads_started":1}
W20260807 11:23:10.801669 19545 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52744
W20260807 11:23:10.802320 17172 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 19408: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 11:23:10.802510 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 19408
I20260807 11:23:10.822800 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 19005
I20260807 11:23:10.824106 19081 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:10.944893 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 19005
2026-08-07T11:23:10Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8802 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:23:11 dist-test-slave-2w3w krb5kdc[19728](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:23:11 dist-test-slave-2w3w krb5kdc[19728](info): set up 2 sockets
Aug 07 11:23:11 dist-test-slave-2w3w krb5kdc[19728](info): commencing operation
krb5kdc: starting...
W20260807 11:23:12.080474 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:23:12 dist-test-slave-2w3w krb5kdc[19728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101792, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:23:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:23:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:23:12.287598 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:38197
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:46287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:38197
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:12.612044 19742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:12.612339 19742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:12.612520 19742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:12.619973 19742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:23:12.620103 19742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:12.620180 19742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:12.620270 19742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:23:12.620332 19742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:23:12.630363 19742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:46287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:38197
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:38197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:12.631796 19742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:12.633229 19742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:12.645670 19748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:12.645784 19751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:12.645721 19747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:12.646482 19742 server_base.cc:1034] running on GCE node
I20260807 11:23:12.647400 19742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:12.648909 19742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:12.650133 19742 hybrid_clock.cc:648] HybridClock initialized: now 1786101792650090 us; error 28 us; skew 500 ppm
I20260807 11:23:12.650499 19742 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:12 dist-test-slave-2w3w krb5kdc[19728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101792, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:12.654162 19742 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:23:12.655817 19742 webserver.cc:460] Webserver started at http://127.16.197.62:37499/ using document root <none> and password file <none>
I20260807 11:23:12.656580 19742 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:12.656683 19742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:12.656956 19742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:12.660015 19742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "adcd67298c23445ca17b454cdcce6912"
format_stamp: "Formatted at 2026-08-07 11:23:12 on dist-test-slave-2w3w"
server_key: "46148ed181aa79cb5d7acd2a720e8e76"
server_key_iv: "90a3cb68611f416991d148e5913c9e2b"
server_key_version: "encryptionkey@0"
I20260807 11:23:12.660735 19742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "adcd67298c23445ca17b454cdcce6912"
format_stamp: "Formatted at 2026-08-07 11:23:12 on dist-test-slave-2w3w"
server_key: "46148ed181aa79cb5d7acd2a720e8e76"
server_key_iv: "90a3cb68611f416991d148e5913c9e2b"
server_key_version: "encryptionkey@0"
I20260807 11:23:12.665854 19742 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 11:23:12.669345 19758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:12.670706 19742 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260807 11:23:12.670953 19742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "adcd67298c23445ca17b454cdcce6912"
format_stamp: "Formatted at 2026-08-07 11:23:12 on dist-test-slave-2w3w"
server_key: "46148ed181aa79cb5d7acd2a720e8e76"
server_key_iv: "90a3cb68611f416991d148e5913c9e2b"
server_key_version: "encryptionkey@0"
I20260807 11:23:12.671233 19742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:12.684614 19742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:12.689208 19742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:12.689536 19742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:12.707473 19742 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:38197
I20260807 11:23:12.707480 19811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:38197 every 8 connection(s)
I20260807 11:23:12.708998 19742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:23:12.712853 19812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:12.717792 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 19742
I20260807 11:23:12.718051 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:23:12.718885 17172 external_mini_cluster.cc:1428] Setting key 6c3ea4fbab8053e17750e7005824a45c
I20260807 11:23:12.726280 19812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adcd67298c23445ca17b454cdcce6912: Bootstrap starting.
I20260807 11:23:12.730032 19812 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P adcd67298c23445ca17b454cdcce6912: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:12.731383 19812 log.cc:824] T 00000000000000000000000000000000 P adcd67298c23445ca17b454cdcce6912: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:12.734366 19812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adcd67298c23445ca17b454cdcce6912: No bootstrap required, opened a new log
Aug 07 11:23:12 dist-test-slave-2w3w krb5kdc[19728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101792, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:12.744616 19816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:12.721068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33164 (local address 127.16.197.62:38197)
0807 11:23:12.721882 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:12.721901 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:12.721956 (+ 55us) server_negotiation.cc:407] Connection header received
0807 11:23:12.724573 (+ 2617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:12.724603 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:12.725119 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:12.725549 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:12.726954 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:12.729071 (+ 2117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:12.730442 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:12.731568 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:12.731754 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:12.735892 (+ 4138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:12.735903 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:12.735924 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:12.735996 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:12.740061 (+ 4065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:12.740764 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:12.740772 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:12.740783 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:12.741008 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:12.741383 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:12.741388 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:12.741392 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:12.742162 (+ 770us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:12.742464 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:12.742752 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 11:23:12.743234 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":324,"threads_started":1}
I20260807 11:23:12.745515 19812 raft_consensus.cc:348] T 00000000000000000000000000000000 P adcd67298c23445ca17b454cdcce6912 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adcd67298c23445ca17b454cdcce6912" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 38197 } }
I20260807 11:23:12.745960 19812 raft_consensus.cc:374] T 00000000000000000000000000000000 P adcd67298c23445ca17b454cdcce6912 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:12.746076 19812 raft_consensus.cc:729] T 00000000000000000000000000000000 P adcd67298c23445ca17b454cdcce6912 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adcd67298c23445ca17b454cdcce6912, State: Initialized, Role: FOLLOWER
I20260807 11:23:12.746768 19812 consensus_queue.cc:260] T 00000000000000000000000000000000 P adcd67298c23445ca17b454cdcce6912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adcd67298c23445ca17b454cdcce6912" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 38197 } }
I20260807 11:23:12.747090 19812 raft_consensus.cc:388] T 00000000000000000000000000000000 P adcd67298c23445ca17b454cdcce6912 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:23:12.747243 19812 raft_consensus.cc:482] T 00000000000000000000000000000000 P adcd67298c23445ca17b454cdcce6912 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:23:12.747439 19812 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adcd67298c23445ca17b454cdcce6912 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:12.749644 19812 raft_consensus.cc:504] T 00000000000000000000000000000000 P adcd67298c23445ca17b454cdcce6912 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adcd67298c23445ca17b454cdcce6912" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 38197 } }
I20260807 11:23:12.750150 19812 leader_election.cc:302] T 00000000000000000000000000000000 P adcd67298c23445ca17b454cdcce6912 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: adcd67298c23445ca17b454cdcce6912; no voters:
I20260807 11:23:12.750836 19812 leader_election.cc:288] T 00000000000000000000000000000000 P adcd67298c23445ca17b454cdcce6912 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:23:12.750954 19818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P adcd67298c23445ca17b454cdcce6912 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:12.751684 19818 raft_consensus.cc:686] T 00000000000000000000000000000000 P adcd67298c23445ca17b454cdcce6912 [term 1 LEADER]: Becoming Leader. State: Replica: adcd67298c23445ca17b454cdcce6912, State: Running, Role: LEADER
I20260807 11:23:12.752233 19818 consensus_queue.cc:237] T 00000000000000000000000000000000 P adcd67298c23445ca17b454cdcce6912 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adcd67298c23445ca17b454cdcce6912" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 38197 } }
I20260807 11:23:12.753099 19812 sys_catalog.cc:564] T 00000000000000000000000000000000 P adcd67298c23445ca17b454cdcce6912 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:23:12.755429 19819 sys_catalog.cc:455] T 00000000000000000000000000000000 P adcd67298c23445ca17b454cdcce6912 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "adcd67298c23445ca17b454cdcce6912" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adcd67298c23445ca17b454cdcce6912" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 38197 } } }
I20260807 11:23:12.755415 19820 sys_catalog.cc:455] T 00000000000000000000000000000000 P adcd67298c23445ca17b454cdcce6912 [sys.catalog]: SysCatalogTable state changed. Reason: New leader adcd67298c23445ca17b454cdcce6912. Latest consensus state: current_term: 1 leader_uuid: "adcd67298c23445ca17b454cdcce6912" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adcd67298c23445ca17b454cdcce6912" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 38197 } } }
I20260807 11:23:12.756028 19820 sys_catalog.cc:458] T 00000000000000000000000000000000 P adcd67298c23445ca17b454cdcce6912 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:12.756024 19819 sys_catalog.cc:458] T 00000000000000000000000000000000 P adcd67298c23445ca17b454cdcce6912 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:12.759706 19824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:23:12.765116 19824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:23:12.774266 19824 catalog_manager.cc:1347] Generated new cluster ID: 70d580e582c74920b11357f693b1d8e6
I20260807 11:23:12.774402 19824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:23:12.796547 19824 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:23:12.798043 19824 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:23:12.809530 19824 catalog_manager.cc:5868] T 00000000000000000000000000000000 P adcd67298c23445ca17b454cdcce6912: Generated new TSK 0
I20260807 11:23:12.810407 19824 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.197.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
I20260807 11:23:12.863767 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:0
--local_ip_for_outbound_sockets=127.16.197.1
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:38197
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:46287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:13.201889 19843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:13.202247 19843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:13.202438 19843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:13.209831 19843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:13.209995 19843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:13.210189 19843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:23:13.220484 19843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:46287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:38197
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:13.221849 19843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:13.223344 19843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:13.236936 19849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:13.236902 19848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:13.236989 19852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:13.237854 19843 server_base.cc:1034] running on GCE node
I20260807 11:23:13.238541 19843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:13.239620 19843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:13.240839 19843 hybrid_clock.cc:648] HybridClock initialized: now 1786101793240788 us; error 44 us; skew 500 ppm
I20260807 11:23:13.241182 19843 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:13 dist-test-slave-2w3w krb5kdc[19728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101793, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:13.244807 19843 init.cc:377] Logged in from keytab as kudu/127.16.197.1@KRBTEST.COM (short username kudu)
I20260807 11:23:13.246428 19843 webserver.cc:460] Webserver started at http://127.16.197.1:33631/ using document root <none> and password file <none>
I20260807 11:23:13.247213 19843 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:13.247354 19843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:13.247682 19843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:13.250779 19843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/data/instance:
uuid: "938dbfb2b3c040a6ac01aefe87338ff8"
format_stamp: "Formatted at 2026-08-07 11:23:13 on dist-test-slave-2w3w"
server_key: "aec8064c0a22d1c985611c0564e16a5f"
server_key_iv: "62264153ef0fd227b732d4d67b3cd363"
server_key_version: "encryptionkey@0"
I20260807 11:23:13.251669 19843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance:
uuid: "938dbfb2b3c040a6ac01aefe87338ff8"
format_stamp: "Formatted at 2026-08-07 11:23:13 on dist-test-slave-2w3w"
server_key: "aec8064c0a22d1c985611c0564e16a5f"
server_key_iv: "62264153ef0fd227b732d4d67b3cd363"
server_key_version: "encryptionkey@0"
I20260807 11:23:13.257431 19843 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260807 11:23:13.261437 19859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:13.262892 19843 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 11:23:13.263096 19843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "938dbfb2b3c040a6ac01aefe87338ff8"
format_stamp: "Formatted at 2026-08-07 11:23:13 on dist-test-slave-2w3w"
server_key: "aec8064c0a22d1c985611c0564e16a5f"
server_key_iv: "62264153ef0fd227b732d4d67b3cd363"
server_key_version: "encryptionkey@0"
I20260807 11:23:13.263350 19843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:13.279474 19843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:13.283617 19843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:13.283907 19843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:13.284930 19843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:23:13.286250 19843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:13.286324 19843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:13.286410 19843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:13.286455 19843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:13.320330 19843 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:39187
I20260807 11:23:13.320390 19979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:39187 every 8 connection(s)
I20260807 11:23:13.321713 19843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:23:13.325131 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 19843
I20260807 11:23:13.325385 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance
I20260807 11:23:13.326241 17172 external_mini_cluster.cc:1428] Setting key 84e22c662008fbe3af4b362f4ecb4075
Aug 07 11:23:13 dist-test-slave-2w3w krb5kdc[19728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101793, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:13.346675 19982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:13.324391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:33575 (local address 127.16.197.62:38197)
0807 11:23:13.324964 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:13.324970 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:13.326088 (+ 1118us) server_negotiation.cc:407] Connection header received
0807 11:23:13.328429 (+ 2341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:13.328433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:13.328505 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:13.328592 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:13.330744 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:13.331519 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:13.334073 (+ 2554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:13.334453 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:13.334552 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:13.338946 (+ 4394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:13.338983 (+ 37us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:13.338992 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:13.339038 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:13.342800 (+ 3762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:13.343650 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:13.343659 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:13.343665 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:13.343818 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:13.344346 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:13.344354 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:13.344359 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:13.344757 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:13.344954 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:13.345846 (+ 892us) server_negotiation.cc:299] Negotiation successful
0807 11:23:13.346138 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":259,"threads_started":1}
I20260807 11:23:13.347345 19983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:13.324960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:38197 (local address 127.16.197.1:33575)
0807 11:23:13.325886 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:13.325946 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:13.328145 (+ 2199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:13.328892 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:13.328908 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:13.329708 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:13.330562 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:13.330593 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:13.331697 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:13.331706 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:13.333916 (+ 2210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:13.333935 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:13.334680 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:13.334689 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:13.335069 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:13.335910 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:13.335933 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:13.338706 (+ 2773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:13.343026 (+ 4320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:13.343034 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:13.343049 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:13.343510 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:13.343981 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:13.343985 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:13.343988 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:13.344197 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:13.344915 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:13.344922 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:13.345385 (+ 463us) client_negotiation.cc:769] Sending connection context
0807 11:23:13.345662 (+ 277us) client_negotiation.cc:240] Negotiation successful
0807 11:23:13.346111 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":378,"threads_started":1}
I20260807 11:23:13.349040 19980 heartbeater.cc:347] Connected to a master server at 127.16.197.62:38197
I20260807 11:23:13.349365 19980 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.16.197.2@KRBTEST.COM; defaulting to no policy
I20260807 11:23:13.350147 19980 heartbeater.cc:511] Master 127.16.197.62:38197 requested a full tablet report, sending...
I20260807 11:23:13.352295 19776 ts_manager.cc:194] Registered new tserver with Master: 938dbfb2b3c040a6ac01aefe87338ff8 (127.16.197.1:39187)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.2@KRBTEST.COM" created.
I20260807 11:23:13.354394 19776 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.1@KRBTEST.COM'} at 127.16.197.1:33575
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
I20260807 11:23:13.379562 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:0
--local_ip_for_outbound_sockets=127.16.197.2
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:38197
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:46287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:13.711755 19986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:13.712126 19986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:13.712277 19986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:13.723663 19986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:13.723847 19986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:13.724030 19986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:23:13.734237 19986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:46287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:38197
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:13.735678 19986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:13.737061 19986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:13.750519 19991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:13.750586 19992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:13.750661 19994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:13.751538 19986 server_base.cc:1034] running on GCE node
I20260807 11:23:13.752102 19986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:13.753018 19986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:13.754237 19986 hybrid_clock.cc:648] HybridClock initialized: now 1786101793754170 us; error 47 us; skew 500 ppm
I20260807 11:23:13.754619 19986 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:13 dist-test-slave-2w3w krb5kdc[19728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101793, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:13.765554 19986 init.cc:377] Logged in from keytab as kudu/127.16.197.2@KRBTEST.COM (short username kudu)
I20260807 11:23:13.767257 19986 webserver.cc:460] Webserver started at http://127.16.197.2:34497/ using document root <none> and password file <none>
I20260807 11:23:13.767966 19986 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:13.768067 19986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:13.768396 19986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:13.771502 19986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/data/instance:
uuid: "341baba5d5f948a59e2e41619918251d"
format_stamp: "Formatted at 2026-08-07 11:23:13 on dist-test-slave-2w3w"
server_key: "9160574d40932a84f5010d462490ee4a"
server_key_iv: "1799f80846744882ac978883477a5f7d"
server_key_version: "encryptionkey@0"
I20260807 11:23:13.772320 19986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance:
uuid: "341baba5d5f948a59e2e41619918251d"
format_stamp: "Formatted at 2026-08-07 11:23:13 on dist-test-slave-2w3w"
server_key: "9160574d40932a84f5010d462490ee4a"
server_key_iv: "1799f80846744882ac978883477a5f7d"
server_key_version: "encryptionkey@0"
I20260807 11:23:13.777820 19986 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260807 11:23:13.781312 20001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:13.782713 19986 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 11:23:13.782922 19986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "341baba5d5f948a59e2e41619918251d"
format_stamp: "Formatted at 2026-08-07 11:23:13 on dist-test-slave-2w3w"
server_key: "9160574d40932a84f5010d462490ee4a"
server_key_iv: "1799f80846744882ac978883477a5f7d"
server_key_version: "encryptionkey@0"
I20260807 11:23:13.783178 19986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:13.812703 19986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:13.817481 19986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:13.817803 19986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:13.818835 19986 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:23:13.820231 19986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:13.820305 19986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:13.820475 19986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:13.820526 19986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:13.854352 19986 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:41755
I20260807 11:23:13.854355 20118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:41755 every 8 connection(s)
I20260807 11:23:13.855762 19986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:23:13.860670 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 19986
I20260807 11:23:13.860875 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance
I20260807 11:23:13.861471 17172 external_mini_cluster.cc:1428] Setting key bb4a7d676ab900aedf2b276c0ebac460
Aug 07 11:23:13 dist-test-slave-2w3w krb5kdc[19728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101793, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:13.880439 20121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:13.858413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:44137 (local address 127.16.197.62:38197)
0807 11:23:13.858923 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:13.858929 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:13.860664 (+ 1735us) server_negotiation.cc:407] Connection header received
0807 11:23:13.862885 (+ 2221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:13.862891 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:13.863021 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:13.863159 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:13.865317 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:13.866042 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:13.868650 (+ 2608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:13.869133 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:13.869237 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:13.873129 (+ 3892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:13.873138 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:13.873146 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:13.873190 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:13.876923 (+ 3733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:13.877676 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:13.877684 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:13.877690 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:13.877880 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:13.878313 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:13.878321 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:13.878327 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:13.878723 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:13.878896 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:13.879667 (+ 771us) server_negotiation.cc:299] Negotiation successful
0807 11:23:13.879991 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":245,"threads_started":1}
I20260807 11:23:13.881183 20122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:13.859177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:38197 (local address 127.16.197.2:44137)
0807 11:23:13.860218 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:13.860272 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:13.862595 (+ 2323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:13.863409 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:13.863418 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:13.864238 (+ 820us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:13.865120 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:13.865150 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:13.866208 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:13.866217 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:13.868482 (+ 2265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:13.868501 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:13.869377 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:13.869387 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:13.869722 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:13.870475 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:13.870510 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:13.872892 (+ 2382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:13.877079 (+ 4187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:13.877087 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:13.877101 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:13.877536 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:13.877999 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:13.878002 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:13.878005 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:13.878185 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:13.878861 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:13.878867 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:13.879252 (+ 385us) client_negotiation.cc:769] Sending connection context
0807 11:23:13.879478 (+ 226us) client_negotiation.cc:240] Negotiation successful
0807 11:23:13.880019 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"thread_start_us":403,"threads_started":1}
I20260807 11:23:13.882731 20119 heartbeater.cc:347] Connected to a master server at 127.16.197.62:38197
I20260807 11:23:13.883114 20119 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:13.883853 20119 heartbeater.cc:511] Master 127.16.197.62:38197 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.16.197.3@KRBTEST.COM; defaulting to no policy
I20260807 11:23:13.885499 19776 ts_manager.cc:194] Registered new tserver with Master: 341baba5d5f948a59e2e41619918251d (127.16.197.2:41755)
I20260807 11:23:13.886410 19776 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.2@KRBTEST.COM'} at 127.16.197.2:44137
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
I20260807 11:23:13.913785 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:0
--local_ip_for_outbound_sockets=127.16.197.3
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:38197
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:46287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:14.244186 20127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:14.244455 20127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:14.244625 20127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:14.251971 20127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:14.252099 20127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:14.252290 20127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:23:14.262185 20127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:46287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:38197
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:14.263516 20127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:14.264832 20127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:14.277976 20135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:14.278057 20133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:14.278057 20132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:14.279415 20127 server_base.cc:1034] running on GCE node
I20260807 11:23:14.279987 20127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:14.280853 20127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:14.282056 20127 hybrid_clock.cc:648] HybridClock initialized: now 1786101794282014 us; error 28 us; skew 500 ppm
I20260807 11:23:14.282377 20127 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:14 dist-test-slave-2w3w krb5kdc[19728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101794, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:14.285732 20127 init.cc:377] Logged in from keytab as kudu/127.16.197.3@KRBTEST.COM (short username kudu)
I20260807 11:23:14.287197 20127 webserver.cc:460] Webserver started at http://127.16.197.3:34531/ using document root <none> and password file <none>
I20260807 11:23:14.287846 20127 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:14.287973 20127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:14.288302 20127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:14.291250 20127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/data/instance:
uuid: "b8d01ef5c6a54e84891ed0b36c9650dd"
format_stamp: "Formatted at 2026-08-07 11:23:14 on dist-test-slave-2w3w"
server_key: "53224716ac616f99c136cba87b6e1c3c"
server_key_iv: "3b9180b03f31b946bfa6c0366f75f645"
server_key_version: "encryptionkey@0"
I20260807 11:23:14.292049 20127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance:
uuid: "b8d01ef5c6a54e84891ed0b36c9650dd"
format_stamp: "Formatted at 2026-08-07 11:23:14 on dist-test-slave-2w3w"
server_key: "53224716ac616f99c136cba87b6e1c3c"
server_key_iv: "3b9180b03f31b946bfa6c0366f75f645"
server_key_version: "encryptionkey@0"
I20260807 11:23:14.297308 20127 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 11:23:14.300769 20142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:14.302098 20127 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 11:23:14.302281 20127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "b8d01ef5c6a54e84891ed0b36c9650dd"
format_stamp: "Formatted at 2026-08-07 11:23:14 on dist-test-slave-2w3w"
server_key: "53224716ac616f99c136cba87b6e1c3c"
server_key_iv: "3b9180b03f31b946bfa6c0366f75f645"
server_key_version: "encryptionkey@0"
I20260807 11:23:14.302497 20127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:14.323668 20127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:14.328100 20127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:14.328398 20127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:14.329386 20127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:23:14.330724 20127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:14.330807 20127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:14.330996 20127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:14.331074 20127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260807 11:23:14.357915 19980 heartbeater.cc:503] Master 127.16.197.62:38197 was elected leader, sending a full tablet report...
I20260807 11:23:14.365229 20127 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:44441
I20260807 11:23:14.365315 20260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:44441 every 8 connection(s)
I20260807 11:23:14.366629 20127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:23:14.374560 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 20127
I20260807 11:23:14.374859 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance
I20260807 11:23:14.375419 17172 external_mini_cluster.cc:1428] Setting key 79086d3c864b45b3eb1ce18251443616
Aug 07 11:23:14 dist-test-slave-2w3w krb5kdc[19728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101794, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:14.388098 20121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:14.369588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:44519 (local address 127.16.197.62:38197)
0807 11:23:14.369801 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:14.369808 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:14.371038 (+ 1230us) server_negotiation.cc:407] Connection header received
0807 11:23:14.372890 (+ 1852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:14.372896 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:14.372986 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:14.373111 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:14.374662 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:14.375715 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:14.377882 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:14.378368 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:14.378494 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:14.381985 (+ 3491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:14.381990 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:14.381993 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:14.382027 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:14.384664 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:14.385568 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:14.385574 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:14.385577 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:14.385709 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:14.386177 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:14.386183 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:14.386186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:14.386490 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:14.386665 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:14.387491 (+ 826us) server_negotiation.cc:299] Negotiation successful
0807 11:23:14.387762 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 11:23:14.389257 20263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:14.370117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:38197 (local address 127.16.197.3:44519)
0807 11:23:14.370839 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:14.370891 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:14.372618 (+ 1727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:14.373341 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:14.373349 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:14.373858 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:14.374487 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:14.374508 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:14.375874 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:14.375886 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:14.377715 (+ 1829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:14.377733 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:14.378603 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:14.378614 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:14.378835 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:14.379509 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:14.379531 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:14.381725 (+ 2194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:14.384931 (+ 3206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:14.384941 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:14.384959 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:14.385436 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:14.385842 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:14.385847 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:14.385851 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:14.386056 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:14.386630 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:14.386638 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:14.387078 (+ 440us) client_negotiation.cc:769] Sending connection context
0807 11:23:14.387378 (+ 300us) client_negotiation.cc:240] Negotiation successful
0807 11:23:14.387947 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":287,"threads_started":1}
I20260807 11:23:14.391094 20261 heartbeater.cc:347] Connected to a master server at 127.16.197.62:38197
I20260807 11:23:14.391455 20261 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:14.392236 20261 heartbeater.cc:511] Master 127.16.197.62:38197 requested a full tablet report, sending...
I20260807 11:23:14.393584 19776 ts_manager.cc:194] Registered new tserver with Master: b8d01ef5c6a54e84891ed0b36c9650dd (127.16.197.3:44441)
I20260807 11:23:14.394459 19776 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.3@KRBTEST.COM'} at 127.16.197.3:44519
I20260807 11:23:14.403990 17172 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 11:23:14.437317 20121 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:23:14.434788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33180 (local address 127.16.197.62:38197)
0807 11:23:14.435077 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:14.435086 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:14.435412 (+ 326us) server_negotiation.cc:407] Connection header received
0807 11:23:14.435582 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:14.435586 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:14.435654 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:14.435742 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:14.437009 (+ 1267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33180: BlockingRecv error: recv got EOF from 127.0.0.1:33180 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 11:23:14.440749 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 19843
W20260807 11:23:14.443702 17172 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 19843: Not authorized: Client connection negotiation failed: client connection to 127.16.197.1:39187: server requires authentication, but client does not have Kerberos credentials available
I20260807 11:23:14.443863 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 19843
I20260807 11:23:14.461352 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 19986
W20260807 11:23:14.464156 17172 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 19986: Not authorized: Client connection negotiation failed: client connection to 127.16.197.2:41755: server requires authentication, but client does not have Kerberos credentials available
I20260807 11:23:14.464356 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 19986
I20260807 11:23:14.481840 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 20127
W20260807 11:23:14.484774 17172 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 20127: Not authorized: Client connection negotiation failed: client connection to 127.16.197.3:44441: server requires authentication, but client does not have Kerberos credentials available
I20260807 11:23:14.484942 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 20127
I20260807 11:23:14.502051 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 19742
I20260807 11:23:14.503468 19807 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:14.607111 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 19742
2026-08-07T11:23:14Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3658 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:23:14 dist-test-slave-2w3w krb5kdc[20279](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:23:14 dist-test-slave-2w3w krb5kdc[20279](info): set up 2 sockets
Aug 07 11:23:14 dist-test-slave-2w3w krb5kdc[20279](info): commencing operation
krb5kdc: starting...
W20260807 11:23:15.733088 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:23:15 dist-test-slave-2w3w krb5kdc[20279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:23:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:23:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:23:15.946247 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:34031
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:34781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:34031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:16.274570 20294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:16.274977 20294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:16.275139 20294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:16.282449 20294 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:23:16.282575 20294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:16.282682 20294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:16.282789 20294 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:23:16.282873 20294 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:23:16.293018 20294 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:34781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:34031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:34031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:16.294415 20294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:16.295892 20294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:16.308207 20300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:16.308375 20299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:16.308297 20302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:16.309474 20294 server_base.cc:1034] running on GCE node
I20260807 11:23:16.310278 20294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:16.311779 20294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:16.312980 20294 hybrid_clock.cc:648] HybridClock initialized: now 1786101796312924 us; error 50 us; skew 500 ppm
I20260807 11:23:16.313344 20294 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:16 dist-test-slave-2w3w krb5kdc[20279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101796, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:16.316676 20294 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:23:16.318293 20294 webserver.cc:460] Webserver started at http://127.16.197.62:40779/ using document root <none> and password file <none>
I20260807 11:23:16.319031 20294 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:16.319147 20294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:16.319423 20294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:16.322417 20294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "8fc6f5161c754e9fb3d2fee4c17df478"
format_stamp: "Formatted at 2026-08-07 11:23:16 on dist-test-slave-2w3w"
server_key: "a749b0b9f821955a580f8c7708c21e03"
server_key_iv: "256a1da461ac5afde6f9cad8170f1e48"
server_key_version: "encryptionkey@0"
I20260807 11:23:16.323280 20294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "8fc6f5161c754e9fb3d2fee4c17df478"
format_stamp: "Formatted at 2026-08-07 11:23:16 on dist-test-slave-2w3w"
server_key: "a749b0b9f821955a580f8c7708c21e03"
server_key_iv: "256a1da461ac5afde6f9cad8170f1e48"
server_key_version: "encryptionkey@0"
I20260807 11:23:16.328483 20294 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260807 11:23:16.331861 20310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:16.333190 20294 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 11:23:16.333359 20294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "8fc6f5161c754e9fb3d2fee4c17df478"
format_stamp: "Formatted at 2026-08-07 11:23:16 on dist-test-slave-2w3w"
server_key: "a749b0b9f821955a580f8c7708c21e03"
server_key_iv: "256a1da461ac5afde6f9cad8170f1e48"
server_key_version: "encryptionkey@0"
I20260807 11:23:16.333544 20294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:16.363579 20294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:16.368043 20294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:16.368346 20294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:16.387364 20294 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:34031
I20260807 11:23:16.387413 20362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:34031 every 8 connection(s)
I20260807 11:23:16.388783 20294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:23:16.392620 20363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:16.397205 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 20294
I20260807 11:23:16.397461 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:23:16.398281 17172 external_mini_cluster.cc:1428] Setting key 8d639a93d20bbf707225a65d22e83429
I20260807 11:23:16.404897 20363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fc6f5161c754e9fb3d2fee4c17df478: Bootstrap starting.
I20260807 11:23:16.408700 20363 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fc6f5161c754e9fb3d2fee4c17df478: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:16.409742 20363 log.cc:824] T 00000000000000000000000000000000 P 8fc6f5161c754e9fb3d2fee4c17df478: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:16.412528 20363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fc6f5161c754e9fb3d2fee4c17df478: No bootstrap required, opened a new log
Aug 07 11:23:16 dist-test-slave-2w3w krb5kdc[20279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:16.421017 20363 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fc6f5161c754e9fb3d2fee4c17df478 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fc6f5161c754e9fb3d2fee4c17df478" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 34031 } }
I20260807 11:23:16.421450 20363 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fc6f5161c754e9fb3d2fee4c17df478 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:16.421576 20363 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fc6f5161c754e9fb3d2fee4c17df478 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fc6f5161c754e9fb3d2fee4c17df478, State: Initialized, Role: FOLLOWER
I20260807 11:23:16.421995 20366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:16.400249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48526 (local address 127.16.197.62:34031)
0807 11:23:16.401099 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:16.401111 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:16.401158 (+ 47us) server_negotiation.cc:407] Connection header received
0807 11:23:16.403285 (+ 2127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:16.403307 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:16.403704 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:16.404028 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:16.405932 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:16.407341 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:16.409214 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:16.409976 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:16.410180 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:16.414064 (+ 3884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:16.414072 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:16.414085 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:16.414125 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:16.417410 (+ 3285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:16.418158 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:16.418178 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:16.418185 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:16.418353 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:16.418792 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:16.418797 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:16.418800 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:16.419494 (+ 694us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:16.419821 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:16.420310 (+ 489us) server_negotiation.cc:299] Negotiation successful
0807 11:23:16.420741 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":418,"threads_started":1}
I20260807 11:23:16.422243 20363 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fc6f5161c754e9fb3d2fee4c17df478 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fc6f5161c754e9fb3d2fee4c17df478" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 34031 } }
I20260807 11:23:16.422506 20363 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8fc6f5161c754e9fb3d2fee4c17df478 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:23:16.422667 20363 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fc6f5161c754e9fb3d2fee4c17df478 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:23:16.422884 20363 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fc6f5161c754e9fb3d2fee4c17df478 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:16.425686 20363 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fc6f5161c754e9fb3d2fee4c17df478 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fc6f5161c754e9fb3d2fee4c17df478" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 34031 } }
I20260807 11:23:16.426218 20363 leader_election.cc:302] T 00000000000000000000000000000000 P 8fc6f5161c754e9fb3d2fee4c17df478 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8fc6f5161c754e9fb3d2fee4c17df478; no voters:
I20260807 11:23:16.426501 17172 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 11:23:16.427062 20363 leader_election.cc:288] T 00000000000000000000000000000000 P 8fc6f5161c754e9fb3d2fee4c17df478 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:23:16.427160 20368 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fc6f5161c754e9fb3d2fee4c17df478 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:16.429703 20363 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fc6f5161c754e9fb3d2fee4c17df478 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:23:16.434103 20368 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fc6f5161c754e9fb3d2fee4c17df478 [term 1 LEADER]: Becoming Leader. State: Replica: 8fc6f5161c754e9fb3d2fee4c17df478, State: Running, Role: LEADER
I20260807 11:23:16.434864 20368 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fc6f5161c754e9fb3d2fee4c17df478 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fc6f5161c754e9fb3d2fee4c17df478" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 34031 } }
I20260807 11:23:16.440778 20369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fc6f5161c754e9fb3d2fee4c17df478 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8fc6f5161c754e9fb3d2fee4c17df478" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fc6f5161c754e9fb3d2fee4c17df478" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 34031 } } }
I20260807 11:23:16.441241 20369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fc6f5161c754e9fb3d2fee4c17df478 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:16.440765 20370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fc6f5161c754e9fb3d2fee4c17df478 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fc6f5161c754e9fb3d2fee4c17df478. Latest consensus state: current_term: 1 leader_uuid: "8fc6f5161c754e9fb3d2fee4c17df478" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fc6f5161c754e9fb3d2fee4c17df478" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 34031 } } }
I20260807 11:23:16.442988 20370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fc6f5161c754e9fb3d2fee4c17df478 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:16.443861 20381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:23:16.449512 20381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:23:16.458811 20381 catalog_manager.cc:1347] Generated new cluster ID: d9c710ee441c4d0ba30bf32e374bcce3
I20260807 11:23:16.459035 20381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:23:16.481904 20366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:16.472698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48528 (local address 127.16.197.62:34031)
0807 11:23:16.472899 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:16.472910 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:16.473417 (+ 507us) server_negotiation.cc:407] Connection header received
0807 11:23:16.473589 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:16.473595 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:16.473685 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:16.473799 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:16.475036 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:16.476217 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:16.478079 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:16.478736 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:16.478884 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:16.480096 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:16.480103 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:16.480108 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:23:16.480988 (+ 880us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:16.481067 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:23:16.481103 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:16.481217 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:16.481367 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 11:23:16.481486 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 11:23:16.486672 20381 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:23:16.488521 20381 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:23:16.497720 20381 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fc6f5161c754e9fb3d2fee4c17df478: Generated new TSK 0
I20260807 11:23:16.498661 20381 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:23:16.509714 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 20294
I20260807 11:23:16.511263 20358 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:16.601861 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 20294
2026-08-07T11:23:16Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1989 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:23:16 dist-test-slave-2w3w krb5kdc[20398](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:23:16 dist-test-slave-2w3w krb5kdc[20398](info): set up 2 sockets
Aug 07 11:23:16 dist-test-slave-2w3w krb5kdc[20398](info): commencing operation
krb5kdc: starting...
W20260807 11:23:17.809073 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.002s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:23:17 dist-test-slave-2w3w krb5kdc[20398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:23:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:23:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:23:18.022939 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:46507
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:38083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:46507
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:18.347685 20411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:18.348044 20411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:18.348201 20411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:18.355541 20411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:23:18.355683 20411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:18.355796 20411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:18.355901 20411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:23:18.355990 20411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:23:18.365928 20411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:38083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:46507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:46507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:18.367401 20411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:18.368873 20411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:18.381116 20416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:18.381048 20417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:18.381089 20419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:18.382130 20411 server_base.cc:1034] running on GCE node
I20260807 11:23:18.382913 20411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:18.384470 20411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:18.385682 20411 hybrid_clock.cc:648] HybridClock initialized: now 1786101798385632 us; error 39 us; skew 500 ppm
I20260807 11:23:18.386067 20411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:18 dist-test-slave-2w3w krb5kdc[20398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101798, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:18.389240 20411 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:23:18.390786 20411 webserver.cc:460] Webserver started at http://127.16.197.62:34331/ using document root <none> and password file <none>
I20260807 11:23:18.391618 20411 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:18.391763 20411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:18.392096 20411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:18.395211 20411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "7fb746d5884348e9a503c69305f3b0d3"
format_stamp: "Formatted at 2026-08-07 11:23:18 on dist-test-slave-2w3w"
server_key: "0b0d9bcf7ce73335323a3e4bc1755a79"
server_key_iv: "b1ea1e5a6bbc609f48a255a84a2a981a"
server_key_version: "encryptionkey@0"
I20260807 11:23:18.396034 20411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "7fb746d5884348e9a503c69305f3b0d3"
format_stamp: "Formatted at 2026-08-07 11:23:18 on dist-test-slave-2w3w"
server_key: "0b0d9bcf7ce73335323a3e4bc1755a79"
server_key_iv: "b1ea1e5a6bbc609f48a255a84a2a981a"
server_key_version: "encryptionkey@0"
I20260807 11:23:18.401376 20411 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 11:23:18.404939 20429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:18.406289 20411 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 11:23:18.406486 20411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "7fb746d5884348e9a503c69305f3b0d3"
format_stamp: "Formatted at 2026-08-07 11:23:18 on dist-test-slave-2w3w"
server_key: "0b0d9bcf7ce73335323a3e4bc1755a79"
server_key_iv: "b1ea1e5a6bbc609f48a255a84a2a981a"
server_key_version: "encryptionkey@0"
I20260807 11:23:18.406684 20411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:18.441571 20411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:18.445735 20411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:18.446049 20411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:18.465003 20411 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:46507
I20260807 11:23:18.465052 20482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:46507 every 8 connection(s)
I20260807 11:23:18.466410 20411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:23:18.470158 20483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:18.472802 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 20411
I20260807 11:23:18.473013 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:23:18.473635 17172 external_mini_cluster.cc:1428] Setting key 2127b1e556cd191f18101461eb5f7053
I20260807 11:23:18.482987 20483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fb746d5884348e9a503c69305f3b0d3: Bootstrap starting.
I20260807 11:23:18.486449 20483 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7fb746d5884348e9a503c69305f3b0d3: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:18.487525 20483 log.cc:824] T 00000000000000000000000000000000 P 7fb746d5884348e9a503c69305f3b0d3: Log is configured to *not* fsync() on all Append() calls
Aug 07 11:23:18 dist-test-slave-2w3w krb5kdc[20398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:18.490808 20483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fb746d5884348e9a503c69305f3b0d3: No bootstrap required, opened a new log
I20260807 11:23:18.498133 20486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:18.475667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52238 (local address 127.16.197.62:46507)
0807 11:23:18.476473 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:18.476488 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:18.476537 (+ 49us) server_negotiation.cc:407] Connection header received
0807 11:23:18.478730 (+ 2193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:18.478762 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:18.479313 (+ 551us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:18.479768 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:18.481017 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:18.482993 (+ 1976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:18.484496 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:18.485562 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:18.485775 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:18.490554 (+ 4779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:18.490563 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:18.490577 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:18.490612 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:18.493829 (+ 3217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:18.494566 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:18.494573 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:18.494579 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:18.494757 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:18.495238 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:18.495249 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:18.495251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:18.495811 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:18.496148 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:18.496621 (+ 473us) server_negotiation.cc:299] Negotiation successful
0807 11:23:18.497076 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":347,"threads_started":1}
I20260807 11:23:18.499253 20483 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7fb746d5884348e9a503c69305f3b0d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fb746d5884348e9a503c69305f3b0d3" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46507 } }
I20260807 11:23:18.499648 20483 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7fb746d5884348e9a503c69305f3b0d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:18.499792 20483 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7fb746d5884348e9a503c69305f3b0d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fb746d5884348e9a503c69305f3b0d3, State: Initialized, Role: FOLLOWER
I20260807 11:23:18.500355 20483 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7fb746d5884348e9a503c69305f3b0d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fb746d5884348e9a503c69305f3b0d3" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46507 } }
I20260807 11:23:18.500618 20483 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7fb746d5884348e9a503c69305f3b0d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:23:18.500778 20483 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7fb746d5884348e9a503c69305f3b0d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:23:18.500965 20483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7fb746d5884348e9a503c69305f3b0d3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:18.503460 20483 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7fb746d5884348e9a503c69305f3b0d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fb746d5884348e9a503c69305f3b0d3" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46507 } }
I20260807 11:23:18.503989 20483 leader_election.cc:302] T 00000000000000000000000000000000 P 7fb746d5884348e9a503c69305f3b0d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7fb746d5884348e9a503c69305f3b0d3; no voters:
I20260807 11:23:18.504745 20483 leader_election.cc:288] T 00000000000000000000000000000000 P 7fb746d5884348e9a503c69305f3b0d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:23:18.504864 20488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7fb746d5884348e9a503c69305f3b0d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:18.505573 20488 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7fb746d5884348e9a503c69305f3b0d3 [term 1 LEADER]: Becoming Leader. State: Replica: 7fb746d5884348e9a503c69305f3b0d3, State: Running, Role: LEADER
I20260807 11:23:18.506213 20488 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7fb746d5884348e9a503c69305f3b0d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fb746d5884348e9a503c69305f3b0d3" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46507 } }
I20260807 11:23:18.507084 20483 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7fb746d5884348e9a503c69305f3b0d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:23:18.509467 20489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fb746d5884348e9a503c69305f3b0d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7fb746d5884348e9a503c69305f3b0d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fb746d5884348e9a503c69305f3b0d3" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46507 } } }
I20260807 11:23:18.509946 20489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fb746d5884348e9a503c69305f3b0d3 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:18.510494 20490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fb746d5884348e9a503c69305f3b0d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7fb746d5884348e9a503c69305f3b0d3. Latest consensus state: current_term: 1 leader_uuid: "7fb746d5884348e9a503c69305f3b0d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fb746d5884348e9a503c69305f3b0d3" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46507 } } }
I20260807 11:23:18.510856 20490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fb746d5884348e9a503c69305f3b0d3 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:18.515784 20495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:23:18.521808 20495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:23:18.531533 20495 catalog_manager.cc:1347] Generated new cluster ID: 254df5c7d8fb417b83ec07e49541ad09
I20260807 11:23:18.531675 20495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:23:18.550546 20495 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:23:18.551923 20495 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:23:18.558375 20495 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7fb746d5884348e9a503c69305f3b0d3: Generated new TSK 0
I20260807 11:23:18.559551 20495 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.197.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
I20260807 11:23:18.623400 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:0
--local_ip_for_outbound_sockets=127.16.197.1
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:46507
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:38083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:18.975063 20509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:18.975464 20509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:18.975648 20509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:18.983872 20509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:18.984056 20509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:18.984237 20509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:23:18.994431 20509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:38083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:46507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:18.995929 20509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:18.997306 20509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:19.011492 20517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:19.011458 20515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:19.011703 20514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:19.012379 20509 server_base.cc:1034] running on GCE node
I20260807 11:23:19.013103 20509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:19.014106 20509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:19.015329 20509 hybrid_clock.cc:648] HybridClock initialized: now 1786101799015270 us; error 67 us; skew 500 ppm
I20260807 11:23:19.015720 20509 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:19 dist-test-slave-2w3w krb5kdc[20398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101799, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:19.023165 20509 init.cc:377] Logged in from keytab as kudu/127.16.197.1@KRBTEST.COM (short username kudu)
I20260807 11:23:19.024874 20509 webserver.cc:460] Webserver started at http://127.16.197.1:42355/ using document root <none> and password file <none>
I20260807 11:23:19.025557 20509 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:19.025665 20509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:19.025961 20509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:19.029203 20509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/data/instance:
uuid: "bfb8b3743bdd4314855466a75b22bd6b"
format_stamp: "Formatted at 2026-08-07 11:23:19 on dist-test-slave-2w3w"
server_key: "703f16c26b3b53e7467c8598a199d247"
server_key_iv: "13ead6e21aa88a670534a8594782656b"
server_key_version: "encryptionkey@0"
I20260807 11:23:19.029937 20509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance:
uuid: "bfb8b3743bdd4314855466a75b22bd6b"
format_stamp: "Formatted at 2026-08-07 11:23:19 on dist-test-slave-2w3w"
server_key: "703f16c26b3b53e7467c8598a199d247"
server_key_iv: "13ead6e21aa88a670534a8594782656b"
server_key_version: "encryptionkey@0"
I20260807 11:23:19.035355 20509 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 11:23:19.039067 20525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:19.040613 20509 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 11:23:19.040798 20509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "bfb8b3743bdd4314855466a75b22bd6b"
format_stamp: "Formatted at 2026-08-07 11:23:19 on dist-test-slave-2w3w"
server_key: "703f16c26b3b53e7467c8598a199d247"
server_key_iv: "13ead6e21aa88a670534a8594782656b"
server_key_version: "encryptionkey@0"
I20260807 11:23:19.041020 20509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:19.086709 20509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:19.091351 20509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:19.091665 20509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:19.092669 20509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:23:19.094012 20509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:19.094090 20509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:19.094250 20509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:19.094314 20509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:19.129048 20509 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:39395
I20260807 11:23:19.129097 20645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:39395 every 8 connection(s)
I20260807 11:23:19.130438 20509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:23:19.138367 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 20509
I20260807 11:23:19.138608 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance
I20260807 11:23:19.139429 17172 external_mini_cluster.cc:1428] Setting key 5a153ce8411179cd6c56afb28bb3f86d
Aug 07 11:23:19 dist-test-slave-2w3w krb5kdc[20398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101799, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:19.152752 20648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:19.133200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:50675 (local address 127.16.197.62:46507)
0807 11:23:19.133676 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:19.133681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:19.134547 (+ 866us) server_negotiation.cc:407] Connection header received
0807 11:23:19.136535 (+ 1988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:19.136543 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:19.136644 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:19.136775 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:19.138340 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:19.139391 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:19.141281 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:19.141677 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:19.141832 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:19.145842 (+ 4010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:19.145850 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:19.145855 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:19.145897 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:19.149385 (+ 3488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:19.150237 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:19.150243 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:19.150248 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:19.150382 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:19.150852 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:19.150858 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:19.150861 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:19.151211 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:19.151383 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:19.152138 (+ 755us) server_negotiation.cc:299] Negotiation successful
0807 11:23:19.152386 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":259,"threads_started":1}
I20260807 11:23:19.153792 20649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:19.133664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46507 (local address 127.16.197.1:50675)
0807 11:23:19.134371 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:19.134418 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:19.136239 (+ 1821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:19.137011 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:19.137019 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:19.137551 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:19.138151 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:19.138170 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:19.139571 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:19.139576 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:19.141043 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:19.141063 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:19.141939 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:19.141947 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:19.142186 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:19.142942 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:19.143019 (+ 77us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:19.145559 (+ 2540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:19.149571 (+ 4012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:19.149581 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:19.149600 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:19.150082 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:19.150503 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:19.150508 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:19.150512 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:19.150720 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:19.151365 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:19.151373 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:19.151738 (+ 365us) client_negotiation.cc:769] Sending connection context
0807 11:23:19.152025 (+ 287us) client_negotiation.cc:240] Negotiation successful
0807 11:23:19.152602 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":269,"threads_started":1}
I20260807 11:23:19.155516 20646 heartbeater.cc:347] Connected to a master server at 127.16.197.62:46507
I20260807 11:23:19.155869 20646 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:19.156658 20646 heartbeater.cc:511] Master 127.16.197.62:46507 requested a full tablet report, sending...
I20260807 11:23:19.159266 20446 ts_manager.cc:194] Registered new tserver with Master: bfb8b3743bdd4314855466a75b22bd6b (127.16.197.1:39395)
WARNING: no policy specified for kudu/127.16.197.2@KRBTEST.COM; defaulting to no policy
I20260807 11:23:19.161525 20446 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.1@KRBTEST.COM'} at 127.16.197.1:50675
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
I20260807 11:23:19.190361 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:0
--local_ip_for_outbound_sockets=127.16.197.2
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:46507
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:38083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:19.541906 20652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:19.542197 20652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:19.542299 20652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:19.549642 20652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:19.549765 20652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:19.549882 20652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:23:19.559839 20652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:38083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:46507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:19.561282 20652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:19.562649 20652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:19.576323 20659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:19.576361 20658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:19.576294 20661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:19.577246 20652 server_base.cc:1034] running on GCE node
I20260807 11:23:19.577891 20652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:19.578892 20652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:19.580180 20652 hybrid_clock.cc:648] HybridClock initialized: now 1786101799580125 us; error 33 us; skew 500 ppm
I20260807 11:23:19.580550 20652 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:19 dist-test-slave-2w3w krb5kdc[20398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101799, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:19.583942 20652 init.cc:377] Logged in from keytab as kudu/127.16.197.2@KRBTEST.COM (short username kudu)
I20260807 11:23:19.585430 20652 webserver.cc:460] Webserver started at http://127.16.197.2:33487/ using document root <none> and password file <none>
I20260807 11:23:19.586153 20652 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:19.586293 20652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:19.586625 20652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:19.589696 20652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/data/instance:
uuid: "a1556649bd5840fea3b59648f50f0d77"
format_stamp: "Formatted at 2026-08-07 11:23:19 on dist-test-slave-2w3w"
server_key: "2fe09ec3358f398e19f27e25b5f18efc"
server_key_iv: "cb9dd30ca57d27a7660d24c8b2fa361f"
server_key_version: "encryptionkey@0"
I20260807 11:23:19.590481 20652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance:
uuid: "a1556649bd5840fea3b59648f50f0d77"
format_stamp: "Formatted at 2026-08-07 11:23:19 on dist-test-slave-2w3w"
server_key: "2fe09ec3358f398e19f27e25b5f18efc"
server_key_iv: "cb9dd30ca57d27a7660d24c8b2fa361f"
server_key_version: "encryptionkey@0"
I20260807 11:23:19.595811 20652 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260807 11:23:19.599689 20669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:19.601146 20652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 11:23:19.601330 20652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "a1556649bd5840fea3b59648f50f0d77"
format_stamp: "Formatted at 2026-08-07 11:23:19 on dist-test-slave-2w3w"
server_key: "2fe09ec3358f398e19f27e25b5f18efc"
server_key_iv: "cb9dd30ca57d27a7660d24c8b2fa361f"
server_key_version: "encryptionkey@0"
I20260807 11:23:19.601563 20652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:19.632511 20652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:19.637403 20652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:19.637758 20652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:19.638803 20652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:23:19.640266 20652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:19.640342 20652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:19.640503 20652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:19.640592 20652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:19.674363 20652 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:33849
I20260807 11:23:19.674423 20790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:33849 every 8 connection(s)
I20260807 11:23:19.675808 20652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:23:19.683768 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 20652
I20260807 11:23:19.683969 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance
I20260807 11:23:19.684432 17172 external_mini_cluster.cc:1428] Setting key 05cab4e91fa513a433d8540f9fdba4d6
Aug 07 11:23:19 dist-test-slave-2w3w krb5kdc[20398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101799, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:19.696836 20793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:19.678388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:36551 (local address 127.16.197.62:46507)
0807 11:23:19.678881 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:19.678886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:19.679745 (+ 859us) server_negotiation.cc:407] Connection header received
0807 11:23:19.681612 (+ 1867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:19.681618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:19.681715 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:19.681838 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:19.683499 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:19.684365 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:19.686157 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:19.686602 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:19.686746 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:19.690721 (+ 3975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:19.690726 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:19.690729 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:19.690763 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:19.693370 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:19.694354 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:19.694358 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:19.694361 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:19.694503 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:19.695059 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:19.695062 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:19.695064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:19.695335 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:19.695453 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:19.696381 (+ 928us) server_negotiation.cc:299] Negotiation successful
0807 11:23:19.696554 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":267,"threads_started":1}
I20260807 11:23:19.698500 20794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:19.678833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46507 (local address 127.16.197.2:36551)
0807 11:23:19.679578 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:19.679617 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:19.681322 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:19.682093 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:19.682101 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:19.682641 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:19.683296 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:19.683315 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:19.684521 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:19.684528 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:19.685983 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:19.685995 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:19.686843 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:19.686850 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:19.687170 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:19.687939 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:19.687962 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:19.690464 (+ 2502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:19.693543 (+ 3079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:19.693553 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:19.693571 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:19.694188 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:19.694619 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:19.694627 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:19.694633 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:19.694914 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:19.695465 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:19.695476 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:19.695982 (+ 506us) client_negotiation.cc:769] Sending connection context
0807 11:23:19.696304 (+ 322us) client_negotiation.cc:240] Negotiation successful
0807 11:23:19.696946 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":282,"threads_started":1}
I20260807 11:23:19.700127 20791 heartbeater.cc:347] Connected to a master server at 127.16.197.62:46507
I20260807 11:23:19.700480 20791 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:19.701319 20791 heartbeater.cc:511] Master 127.16.197.62:46507 requested a full tablet report, sending...
I20260807 11:23:19.703045 20446 ts_manager.cc:194] Registered new tserver with Master: a1556649bd5840fea3b59648f50f0d77 (127.16.197.2:33849)
I20260807 11:23:19.703899 20446 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.2@KRBTEST.COM'} at 127.16.197.2:36551
WARNING: no policy specified for kudu/127.16.197.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
I20260807 11:23:19.735155 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:0
--local_ip_for_outbound_sockets=127.16.197.3
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:46507
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:38083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:20.075433 20797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:20.075786 20797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:20.075963 20797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:20.083500 20797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:20.083657 20797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:20.083851 20797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:23:20.093878 20797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:38083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:46507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:20.095408 20797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:20.096825 20797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:20.110911 20802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:20.110944 20803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:20.110816 20805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:20.111968 20797 server_base.cc:1034] running on GCE node
I20260807 11:23:20.112651 20797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:20.113691 20797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:20.114933 20797 hybrid_clock.cc:648] HybridClock initialized: now 1786101800114885 us; error 44 us; skew 500 ppm
I20260807 11:23:20.115311 20797 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:20 dist-test-slave-2w3w krb5kdc[20398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101800, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:20.118541 20797 init.cc:377] Logged in from keytab as kudu/127.16.197.3@KRBTEST.COM (short username kudu)
I20260807 11:23:20.120052 20797 webserver.cc:460] Webserver started at http://127.16.197.3:44319/ using document root <none> and password file <none>
I20260807 11:23:20.120767 20797 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:20.120918 20797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:20.121232 20797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:20.124296 20797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/data/instance:
uuid: "d5bb41abdefc4f549c01f5990c5168ec"
format_stamp: "Formatted at 2026-08-07 11:23:20 on dist-test-slave-2w3w"
server_key: "e24830aa984bc2ab1115c4efd0291930"
server_key_iv: "b01df9e9b48e773061e0728018a5a54c"
server_key_version: "encryptionkey@0"
I20260807 11:23:20.125113 20797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance:
uuid: "d5bb41abdefc4f549c01f5990c5168ec"
format_stamp: "Formatted at 2026-08-07 11:23:20 on dist-test-slave-2w3w"
server_key: "e24830aa984bc2ab1115c4efd0291930"
server_key_iv: "b01df9e9b48e773061e0728018a5a54c"
server_key_version: "encryptionkey@0"
I20260807 11:23:20.130687 20797 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 11:23:20.134374 20813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:20.135895 20797 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:23:20.136078 20797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "d5bb41abdefc4f549c01f5990c5168ec"
format_stamp: "Formatted at 2026-08-07 11:23:20 on dist-test-slave-2w3w"
server_key: "e24830aa984bc2ab1115c4efd0291930"
server_key_iv: "b01df9e9b48e773061e0728018a5a54c"
server_key_version: "encryptionkey@0"
I20260807 11:23:20.136292 20797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:20.154386 20797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:20.158506 20797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:20.158823 20797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:20.159896 20797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:23:20.161278 20797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:20.161391 20797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:20.161535 20797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:20.161603 20797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:20.165241 20646 heartbeater.cc:503] Master 127.16.197.62:46507 was elected leader, sending a full tablet report...
I20260807 11:23:20.196051 20797 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:46139
I20260807 11:23:20.196110 20933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:46139 every 8 connection(s)
I20260807 11:23:20.197546 20797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:23:20.207013 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 20797
I20260807 11:23:20.207230 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance
I20260807 11:23:20.207685 17172 external_mini_cluster.cc:1428] Setting key c8621a80b261e8813b3feec5fa03331a
Aug 07 11:23:20 dist-test-slave-2w3w krb5kdc[20398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101800, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:20.218639 20936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:20.200340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:36319 (local address 127.16.197.62:46507)
0807 11:23:20.200913 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:20.200918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:20.201860 (+ 942us) server_negotiation.cc:407] Connection header received
0807 11:23:20.203805 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:20.203811 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:20.203918 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:20.204044 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:20.205594 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:20.206395 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:20.208499 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:20.209075 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:20.209208 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:20.213120 (+ 3912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:20.213125 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:20.213129 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:20.213162 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:20.215747 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:20.216519 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:20.216523 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:20.216525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:20.216658 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:20.217100 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:20.217103 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:20.217105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:20.217415 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:20.217539 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:20.218187 (+ 648us) server_negotiation.cc:299] Negotiation successful
0807 11:23:20.218353 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":268,"threads_started":1}
I20260807 11:23:20.219648 20937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:20.200971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46507 (local address 127.16.197.3:36319)
0807 11:23:20.201682 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:20.201726 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:20.203512 (+ 1786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:20.204303 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:20.204311 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:20.204833 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:20.205416 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:20.205433 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:20.206568 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:20.206573 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:20.208327 (+ 1754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:20.208343 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:20.209318 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:20.209324 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:20.209521 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:20.210360 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:20.210400 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:20.212920 (+ 2520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:20.215889 (+ 2969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:20.215896 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:20.215910 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:20.216396 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:20.216753 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:20.216756 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:20.216758 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:20.217000 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:20.217524 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:20.217529 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:20.217857 (+ 328us) client_negotiation.cc:769] Sending connection context
0807 11:23:20.218082 (+ 225us) client_negotiation.cc:240] Negotiation successful
0807 11:23:20.218483 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":273,"threads_started":1}
I20260807 11:23:20.221127 20934 heartbeater.cc:347] Connected to a master server at 127.16.197.62:46507
I20260807 11:23:20.221432 20934 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:20.222168 20934 heartbeater.cc:511] Master 127.16.197.62:46507 requested a full tablet report, sending...
I20260807 11:23:20.223713 20446 ts_manager.cc:194] Registered new tserver with Master: d5bb41abdefc4f549c01f5990c5168ec (127.16.197.3:46139)
I20260807 11:23:20.224524 20446 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.3@KRBTEST.COM'} at 127.16.197.3:36319
I20260807 11:23:20.236104 17172 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 11:23:20 dist-test-slave-2w3w krb5kdc[20398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101800, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 11:23:20 dist-test-slave-2w3w krb5kdc[20398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101800, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:20.288004 20936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:20.273660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52252 (local address 127.16.197.62:46507)
0807 11:23:20.273907 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:20.273916 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:20.274361 (+ 445us) server_negotiation.cc:407] Connection header received
0807 11:23:20.274590 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:20.274595 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:20.274662 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:20.274799 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:20.276313 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:20.277079 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:20.278350 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:20.278754 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:20.278908 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:20.283019 (+ 4111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:20.283025 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:20.283028 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:20.283064 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:20.285580 (+ 2516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:20.286243 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:20.286246 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:20.286248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:20.286380 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:20.286711 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:20.286714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:20.286716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:20.287121 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:20.287309 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:20.287582 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 11:23:20.287765 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260807 11:23:20.288635 20446 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52252
I20260807 11:23:20.291255 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 20509
Aug 07 11:23:20 dist-test-slave-2w3w krb5kdc[20398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101800, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.16.197.1@KRBTEST.COM
I20260807 11:23:20.306927 20948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:20.292194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41504 (local address 127.16.197.1:39395)
0807 11:23:20.292820 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:20.292827 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:20.292854 (+ 27us) server_negotiation.cc:407] Connection header received
0807 11:23:20.292926 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:20.292934 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:20.293143 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:20.293273 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:20.294365 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:20.295462 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:20.296640 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:20.297194 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:20.297313 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:20.301343 (+ 4030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:20.301356 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:20.301371 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:20.301407 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:20.304205 (+ 2798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:20.304976 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:20.304985 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:20.304988 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:20.305174 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:20.305568 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:20.305572 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:20.305574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:20.305972 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:20.306240 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:20.306440 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 11:23:20.306641 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":325,"threads_started":1}
W20260807 11:23:20.308375 20640 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41504
W20260807 11:23:20.309468 17172 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 20509: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 11:23:20.309700 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 20509
I20260807 11:23:20.327885 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 20652
Aug 07 11:23:20 dist-test-slave-2w3w krb5kdc[20398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101800, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.16.197.2@KRBTEST.COM
I20260807 11:23:20.345213 20949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:20.328873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54648 (local address 127.16.197.2:33849)
0807 11:23:20.329434 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:20.329441 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:20.329467 (+ 26us) server_negotiation.cc:407] Connection header received
0807 11:23:20.329526 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:20.329532 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:20.329745 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:20.329930 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:20.331278 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:20.332310 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:20.334156 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:20.334705 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:20.334823 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:20.339208 (+ 4385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:20.339221 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:20.339236 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:20.339272 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:20.342323 (+ 3051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:20.343185 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:20.343193 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:20.343195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:20.343340 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:20.343682 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:20.343685 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:20.343687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:20.344031 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:20.344259 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:20.344572 (+ 313us) server_negotiation.cc:299] Negotiation successful
0807 11:23:20.344780 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":295,"threads_started":1}
W20260807 11:23:20.346588 20785 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54648
W20260807 11:23:20.347841 17172 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 20652: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 11:23:20.348006 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 20652
I20260807 11:23:20.366410 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 20797
Aug 07 11:23:20 dist-test-slave-2w3w krb5kdc[20398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101800, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.16.197.3@KRBTEST.COM
I20260807 11:23:20.383265 20950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:20.367527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54110 (local address 127.16.197.3:46139)
0807 11:23:20.368135 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:20.368143 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:20.368176 (+ 33us) server_negotiation.cc:407] Connection header received
0807 11:23:20.368237 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:20.368243 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:20.368457 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:20.368640 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:20.370054 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:20.371179 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:20.372488 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:20.373094 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:20.373249 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:20.377684 (+ 4435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:20.377696 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:20.377711 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:20.377746 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:20.380727 (+ 2981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:20.381354 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:20.381360 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:20.381363 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:20.381485 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:20.381799 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:20.381802 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:20.381804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:20.382149 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:20.382378 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:20.382630 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 11:23:20.382827 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":315,"threads_started":1}
W20260807 11:23:20.384577 20929 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54110
W20260807 11:23:20.385762 17172 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 20797: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 11:23:20.385951 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 20797
I20260807 11:23:20.402866 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 20411
I20260807 11:23:20.404194 20478 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:20.511234 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 20411
2026-08-07T11:23:20Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3914 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:23:20 dist-test-slave-2w3w krb5kdc[20954](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:23:20 dist-test-slave-2w3w krb5kdc[20954](info): set up 2 sockets
Aug 07 11:23:20 dist-test-slave-2w3w krb5kdc[20954](info): commencing operation
krb5kdc: starting...
W20260807 11:23:21.645406 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:23:21 dist-test-slave-2w3w krb5kdc[20954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:23:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:23:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:23:21.875238 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:42941
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:34741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:42941
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:22.210253 20968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:22.210572 20968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:22.210729 20968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:22.218233 20968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:23:22.218353 20968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:22.218459 20968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:22.218560 20968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:23:22.218647 20968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:23:22.229092 20968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:34741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:42941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:42941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:22.230508 20968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:22.231986 20968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:22.244385 20974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:22.244330 20976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:22.244659 20973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:22.245143 20968 server_base.cc:1034] running on GCE node
I20260807 11:23:22.245973 20968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:22.247586 20968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:22.248850 20968 hybrid_clock.cc:648] HybridClock initialized: now 1786101802248784 us; error 45 us; skew 500 ppm
I20260807 11:23:22.249214 20968 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:22 dist-test-slave-2w3w krb5kdc[20954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101802, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:22.252665 20968 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:23:22.254351 20968 webserver.cc:460] Webserver started at http://127.16.197.62:45375/ using document root <none> and password file <none>
I20260807 11:23:22.255213 20968 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:22.255376 20968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:22.255714 20968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:22.258867 20968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "05b8c868ed1442b1ab25a9538695d87e"
format_stamp: "Formatted at 2026-08-07 11:23:22 on dist-test-slave-2w3w"
server_key: "2fd27f6ca03d90e3e368695bb06e302a"
server_key_iv: "73fbbe3c6c92778591fcfac9bd1efb96"
server_key_version: "encryptionkey@0"
I20260807 11:23:22.259763 20968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "05b8c868ed1442b1ab25a9538695d87e"
format_stamp: "Formatted at 2026-08-07 11:23:22 on dist-test-slave-2w3w"
server_key: "2fd27f6ca03d90e3e368695bb06e302a"
server_key_iv: "73fbbe3c6c92778591fcfac9bd1efb96"
server_key_version: "encryptionkey@0"
I20260807 11:23:22.265431 20968 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 11:23:22.269160 20984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:22.270498 20968 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 11:23:22.270694 20968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "05b8c868ed1442b1ab25a9538695d87e"
format_stamp: "Formatted at 2026-08-07 11:23:22 on dist-test-slave-2w3w"
server_key: "2fd27f6ca03d90e3e368695bb06e302a"
server_key_iv: "73fbbe3c6c92778591fcfac9bd1efb96"
server_key_version: "encryptionkey@0"
I20260807 11:23:22.270946 20968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:22.291942 20968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:22.296221 20968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:22.296535 20968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:22.316087 20968 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:42941
I20260807 11:23:22.316110 21037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:42941 every 8 connection(s)
I20260807 11:23:22.317607 20968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:23:22.321488 21039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:22.327659 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 20968
I20260807 11:23:22.327935 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:23:22.328763 17172 external_mini_cluster.cc:1428] Setting key 05f855468a17bac9c94243719a441a00
I20260807 11:23:22.333356 21039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05b8c868ed1442b1ab25a9538695d87e: Bootstrap starting.
I20260807 11:23:22.337332 21039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05b8c868ed1442b1ab25a9538695d87e: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:22.338533 21039 log.cc:824] T 00000000000000000000000000000000 P 05b8c868ed1442b1ab25a9538695d87e: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:22.342002 21039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05b8c868ed1442b1ab25a9538695d87e: No bootstrap required, opened a new log
Aug 07 11:23:22 dist-test-slave-2w3w krb5kdc[20954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:22.350742 21039 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05b8c868ed1442b1ab25a9538695d87e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b8c868ed1442b1ab25a9538695d87e" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42941 } }
I20260807 11:23:22.351233 21039 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05b8c868ed1442b1ab25a9538695d87e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:22.351364 21039 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05b8c868ed1442b1ab25a9538695d87e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b8c868ed1442b1ab25a9538695d87e, State: Initialized, Role: FOLLOWER
I20260807 11:23:22.352152 21039 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05b8c868ed1442b1ab25a9538695d87e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b8c868ed1442b1ab25a9538695d87e" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42941 } }
I20260807 11:23:22.352514 21039 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05b8c868ed1442b1ab25a9538695d87e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:23:22.352699 21039 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05b8c868ed1442b1ab25a9538695d87e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:23:22.352908 21039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05b8c868ed1442b1ab25a9538695d87e [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:22.353881 21042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:22.331139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59838 (local address 127.16.197.62:42941)
0807 11:23:22.332003 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:22.332015 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:22.332059 (+ 44us) server_negotiation.cc:407] Connection header received
0807 11:23:22.333916 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:22.333941 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:22.334336 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:22.334665 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:22.336715 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:22.338163 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:22.339883 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:22.340709 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:22.340860 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:22.345748 (+ 4888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:22.345756 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:22.345774 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:22.345826 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:22.349185 (+ 3359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:22.349973 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:22.349980 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:22.349987 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:22.350170 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:22.350599 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:22.350603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:22.350605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:22.351321 (+ 716us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:22.351662 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:22.352106 (+ 444us) server_negotiation.cc:299] Negotiation successful
0807 11:23:22.352551 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":391,"threads_started":1}
I20260807 11:23:22.355652 21039 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05b8c868ed1442b1ab25a9538695d87e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b8c868ed1442b1ab25a9538695d87e" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42941 } }
I20260807 11:23:22.356230 21039 leader_election.cc:302] T 00000000000000000000000000000000 P 05b8c868ed1442b1ab25a9538695d87e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05b8c868ed1442b1ab25a9538695d87e; no voters:
I20260807 11:23:22.356914 21039 leader_election.cc:288] T 00000000000000000000000000000000 P 05b8c868ed1442b1ab25a9538695d87e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:23:22.357081 21044 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05b8c868ed1442b1ab25a9538695d87e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:22.357967 21044 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05b8c868ed1442b1ab25a9538695d87e [term 1 LEADER]: Becoming Leader. State: Replica: 05b8c868ed1442b1ab25a9538695d87e, State: Running, Role: LEADER
I20260807 11:23:22.358548 21044 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05b8c868ed1442b1ab25a9538695d87e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b8c868ed1442b1ab25a9538695d87e" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42941 } }
I20260807 11:23:22.359104 21039 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05b8c868ed1442b1ab25a9538695d87e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:23:22.362116 21046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05b8c868ed1442b1ab25a9538695d87e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05b8c868ed1442b1ab25a9538695d87e. Latest consensus state: current_term: 1 leader_uuid: "05b8c868ed1442b1ab25a9538695d87e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b8c868ed1442b1ab25a9538695d87e" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42941 } } }
I20260807 11:23:22.362694 21045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05b8c868ed1442b1ab25a9538695d87e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05b8c868ed1442b1ab25a9538695d87e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b8c868ed1442b1ab25a9538695d87e" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42941 } } }
I20260807 11:23:22.363297 21045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05b8c868ed1442b1ab25a9538695d87e [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:22.363176 21046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05b8c868ed1442b1ab25a9538695d87e [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:22.367523 21052 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:23:22.373027 21052 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:23:22.383889 21052 catalog_manager.cc:1347] Generated new cluster ID: 68ece6f4d2c149dca012771ee7fb070c
I20260807 11:23:22.384037 21052 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:23:22.394455 21052 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:23:22.395943 21052 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:23:22.402640 21052 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05b8c868ed1442b1ab25a9538695d87e: Generated new TSK 0
I20260807 11:23:22.403595 21052 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.197.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
I20260807 11:23:22.466711 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:0
--local_ip_for_outbound_sockets=127.16.197.1
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:42941
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:34741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:22.827306 21065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:22.827606 21065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:22.827771 21065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:22.835413 21065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:22.835546 21065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:22.835733 21065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:23:22.846300 21065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:34741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:42941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:22.848014 21065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:22.849574 21065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:22.864712 21070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:22.864681 21071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:22.864660 21073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:22.865567 21065 server_base.cc:1034] running on GCE node
I20260807 11:23:22.866215 21065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:22.867357 21065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:22.868608 21065 hybrid_clock.cc:648] HybridClock initialized: now 1786101802868540 us; error 63 us; skew 500 ppm
I20260807 11:23:22.868963 21065 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:22 dist-test-slave-2w3w krb5kdc[20954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101802, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:22.872609 21065 init.cc:377] Logged in from keytab as kudu/127.16.197.1@KRBTEST.COM (short username kudu)
I20260807 11:23:22.874217 21065 webserver.cc:460] Webserver started at http://127.16.197.1:37723/ using document root <none> and password file <none>
I20260807 11:23:22.875133 21065 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:22.875289 21065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:22.875630 21065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:22.878713 21065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-0/data/instance:
uuid: "d0fe0cb8fbda4800931159ac3dcc07dd"
format_stamp: "Formatted at 2026-08-07 11:23:22 on dist-test-slave-2w3w"
server_key: "c0c2f619ba04ca11e89f76b473ecd37a"
server_key_iv: "fd57401a792522f73c6b13da6a5d0d4b"
server_key_version: "encryptionkey@0"
I20260807 11:23:22.879593 21065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance:
uuid: "d0fe0cb8fbda4800931159ac3dcc07dd"
format_stamp: "Formatted at 2026-08-07 11:23:22 on dist-test-slave-2w3w"
server_key: "c0c2f619ba04ca11e89f76b473ecd37a"
server_key_iv: "fd57401a792522f73c6b13da6a5d0d4b"
server_key_version: "encryptionkey@0"
I20260807 11:23:22.884955 21065 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 11:23:22.888649 21080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:22.890154 21065 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260807 11:23:22.890377 21065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "d0fe0cb8fbda4800931159ac3dcc07dd"
format_stamp: "Formatted at 2026-08-07 11:23:22 on dist-test-slave-2w3w"
server_key: "c0c2f619ba04ca11e89f76b473ecd37a"
server_key_iv: "fd57401a792522f73c6b13da6a5d0d4b"
server_key_version: "encryptionkey@0"
I20260807 11:23:22.890641 21065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:22.928911 21065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:22.933564 21065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:22.933840 21065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:22.934851 21065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:23:22.936326 21065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:22.936403 21065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:22.936496 21065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:22.936542 21065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:22.971987 21065 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:45387
I20260807 11:23:22.972087 21197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:45387 every 8 connection(s)
I20260807 11:23:22.973471 21065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:23:22.983487 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 21065
I20260807 11:23:22.983750 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance
I20260807 11:23:22.984421 17172 external_mini_cluster.cc:1428] Setting key eae8dc33902ee03bc2b55c9e59c6f950
Aug 07 11:23:22 dist-test-slave-2w3w krb5kdc[20954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101802, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:22.994889 21201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:22.976420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:35921 (local address 127.16.197.62:42941)
0807 11:23:22.976939 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:22.976944 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:22.977854 (+ 910us) server_negotiation.cc:407] Connection header received
0807 11:23:22.979901 (+ 2047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:22.979907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:22.980008 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:22.980144 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:22.981768 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:22.982558 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:22.984527 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:22.984959 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:22.985112 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:22.988853 (+ 3741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:22.988858 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:22.988861 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:22.988895 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:22.991609 (+ 2714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:22.992422 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:22.992426 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:22.992429 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:22.992550 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:22.993023 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:22.993027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:22.993029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:22.993295 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:22.993460 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:22.994226 (+ 766us) server_negotiation.cc:299] Negotiation successful
0807 11:23:22.994481 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":282,"threads_started":1}
I20260807 11:23:22.996052 21202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:22.976952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42941 (local address 127.16.197.1:35921)
0807 11:23:22.977677 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:22.977725 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:22.979597 (+ 1872us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:22.980400 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:22.980409 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:22.980970 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:22.981569 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:22.981587 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:22.982751 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:22.982755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:22.984366 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:22.984380 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:22.985219 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:22.985229 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:22.985535 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:22.986293 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:22.986323 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:22.988613 (+ 2290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:22.991762 (+ 3149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:22.991771 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:22.991790 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:22.992251 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:22.992674 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:22.992679 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:22.992683 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:22.992886 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:22.993435 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:22.993442 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:22.993812 (+ 370us) client_negotiation.cc:769] Sending connection context
0807 11:23:22.994110 (+ 298us) client_negotiation.cc:240] Negotiation successful
0807 11:23:22.994731 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":294,"threads_started":1}
I20260807 11:23:22.997709 21199 heartbeater.cc:347] Connected to a master server at 127.16.197.62:42941
I20260807 11:23:22.998039 21199 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:22.998780 21199 heartbeater.cc:511] Master 127.16.197.62:42941 requested a full tablet report, sending...
I20260807 11:23:23.000993 21001 ts_manager.cc:194] Registered new tserver with Master: d0fe0cb8fbda4800931159ac3dcc07dd (127.16.197.1:45387)
I20260807 11:23:23.002511 21001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.1@KRBTEST.COM'} at 127.16.197.1:35921
WARNING: no policy specified for kudu/127.16.197.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
I20260807 11:23:23.034294 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:0
--local_ip_for_outbound_sockets=127.16.197.2
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:42941
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:34741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:23.378182 21205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:23.378489 21205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:23.378655 21205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:23.385972 21205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:23.386098 21205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:23.386317 21205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:23:23.396675 21205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:34741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:42941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:23.398223 21205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:23.399623 21205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:23.413538 21210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:23.413607 21211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:23.413651 21213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:23.414543 21205 server_base.cc:1034] running on GCE node
I20260807 11:23:23.415247 21205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:23.416157 21205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:23.417356 21205 hybrid_clock.cc:648] HybridClock initialized: now 1786101803417319 us; error 30 us; skew 500 ppm
I20260807 11:23:23.417670 21205 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:23 dist-test-slave-2w3w krb5kdc[20954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101803, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:23.420811 21205 init.cc:377] Logged in from keytab as kudu/127.16.197.2@KRBTEST.COM (short username kudu)
I20260807 11:23:23.422199 21205 webserver.cc:460] Webserver started at http://127.16.197.2:43711/ using document root <none> and password file <none>
I20260807 11:23:23.422883 21205 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:23.423048 21205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:23.423342 21205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:23.426292 21205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-1/data/instance:
uuid: "22bad6f479354fe6a3f9602cec667686"
format_stamp: "Formatted at 2026-08-07 11:23:23 on dist-test-slave-2w3w"
server_key: "26d1be3436cdb1d284b2c57fec7c45c6"
server_key_iv: "24f429e1ea07f8f570377d00000de641"
server_key_version: "encryptionkey@0"
I20260807 11:23:23.427140 21205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance:
uuid: "22bad6f479354fe6a3f9602cec667686"
format_stamp: "Formatted at 2026-08-07 11:23:23 on dist-test-slave-2w3w"
server_key: "26d1be3436cdb1d284b2c57fec7c45c6"
server_key_iv: "24f429e1ea07f8f570377d00000de641"
server_key_version: "encryptionkey@0"
I20260807 11:23:23.432446 21205 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260807 11:23:23.435990 21220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:23.437320 21205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 11:23:23.437461 21205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "22bad6f479354fe6a3f9602cec667686"
format_stamp: "Formatted at 2026-08-07 11:23:23 on dist-test-slave-2w3w"
server_key: "26d1be3436cdb1d284b2c57fec7c45c6"
server_key_iv: "24f429e1ea07f8f570377d00000de641"
server_key_version: "encryptionkey@0"
I20260807 11:23:23.437640 21205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:23.457675 21205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:23.462365 21205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:23.462628 21205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:23.463617 21205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:23:23.464941 21205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:23.465021 21205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:23.465114 21205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:23.465158 21205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:23.500154 21205 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:34591
I20260807 11:23:23.500211 21339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:34591 every 8 connection(s)
I20260807 11:23:23.501523 21205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:23:23.510447 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 21205
I20260807 11:23:23.510679 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance
I20260807 11:23:23.511170 17172 external_mini_cluster.cc:1428] Setting key 0cfb941e1ce79bf8ae98ef55c6566fec
Aug 07 11:23:23 dist-test-slave-2w3w krb5kdc[20954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101803, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:23.523500 21342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:23.504100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:40873 (local address 127.16.197.62:42941)
0807 11:23:23.504594 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:23.504598 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:23.505470 (+ 872us) server_negotiation.cc:407] Connection header received
0807 11:23:23.507454 (+ 1984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:23.507460 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:23.507553 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:23.507679 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:23.509224 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:23.509986 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:23.511840 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:23.512342 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:23.512448 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:23.516898 (+ 4450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:23.516903 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:23.516907 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:23.516941 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:23.519966 (+ 3025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:23.520790 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:23.520799 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:23.520804 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:23.520997 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:23.521428 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:23.521433 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:23.521436 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:23.521844 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:23.522011 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:23.522705 (+ 694us) server_negotiation.cc:299] Negotiation successful
0807 11:23:23.523015 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":273,"threads_started":1}
I20260807 11:23:23.524168 21343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:23.504541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42941 (local address 127.16.197.2:40873)
0807 11:23:23.505277 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:23.505317 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:23.507167 (+ 1850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:23.507945 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:23.507954 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:23.508464 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:23.509037 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:23.509055 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:23.510168 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:23.510173 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:23.511660 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:23.511673 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:23.512580 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:23.512588 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:23.512892 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:23.513733 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:23.513758 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:23.516621 (+ 2863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:23.520135 (+ 3514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:23.520147 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:23.520171 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:23.520637 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:23.521112 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:23.521116 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:23.521118 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:23.521299 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:23.521965 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:23.521971 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:23.522322 (+ 351us) client_negotiation.cc:769] Sending connection context
0807 11:23:23.522557 (+ 235us) client_negotiation.cc:240] Negotiation successful
0807 11:23:23.523027 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":291,"threads_started":1}
I20260807 11:23:23.525739 21340 heartbeater.cc:347] Connected to a master server at 127.16.197.62:42941
I20260807 11:23:23.526068 21340 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:23.526806 21340 heartbeater.cc:511] Master 127.16.197.62:42941 requested a full tablet report, sending...
I20260807 11:23:23.528421 21001 ts_manager.cc:194] Registered new tserver with Master: 22bad6f479354fe6a3f9602cec667686 (127.16.197.2:34591)
I20260807 11:23:23.529387 21001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.2@KRBTEST.COM'} at 127.16.197.2:40873
WARNING: no policy specified for kudu/127.16.197.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
I20260807 11:23:23.562845 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:0
--local_ip_for_outbound_sockets=127.16.197.3
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:42941
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:34741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:23.900645 21346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:23.901042 21346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:23.901206 21346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:23.908797 21346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:23.908962 21346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:23.909147 21346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:23:23.919339 21346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:34741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:42941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:23.920784 21346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:23.922180 21346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:23.935925 21351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:23.936177 21352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:23.936189 21354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:23.937611 21346 server_base.cc:1034] running on GCE node
I20260807 11:23:23.938261 21346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:23.939345 21346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:23.940579 21346 hybrid_clock.cc:648] HybridClock initialized: now 1786101803940504 us; error 60 us; skew 500 ppm
I20260807 11:23:23.940956 21346 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:23 dist-test-slave-2w3w krb5kdc[20954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101803, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:23.944392 21346 init.cc:377] Logged in from keytab as kudu/127.16.197.3@KRBTEST.COM (short username kudu)
I20260807 11:23:23.945977 21346 webserver.cc:460] Webserver started at http://127.16.197.3:41645/ using document root <none> and password file <none>
I20260807 11:23:23.946696 21346 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:23.946836 21346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:23.947211 21346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:23.950208 21346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-2/data/instance:
uuid: "80414fdfacc64775b0039a36d8bfb17c"
format_stamp: "Formatted at 2026-08-07 11:23:23 on dist-test-slave-2w3w"
server_key: "e3a484e27c23c50e4e7fa7280683fe6e"
server_key_iv: "90c6c9373e4d0135322d0c08cf6b220e"
server_key_version: "encryptionkey@0"
I20260807 11:23:23.951088 21346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance:
uuid: "80414fdfacc64775b0039a36d8bfb17c"
format_stamp: "Formatted at 2026-08-07 11:23:23 on dist-test-slave-2w3w"
server_key: "e3a484e27c23c50e4e7fa7280683fe6e"
server_key_iv: "90c6c9373e4d0135322d0c08cf6b220e"
server_key_version: "encryptionkey@0"
I20260807 11:23:23.956350 21346 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260807 11:23:23.959956 21361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:23.961336 21346 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 11:23:23.961512 21346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "80414fdfacc64775b0039a36d8bfb17c"
format_stamp: "Formatted at 2026-08-07 11:23:23 on dist-test-slave-2w3w"
server_key: "e3a484e27c23c50e4e7fa7280683fe6e"
server_key_iv: "90c6c9373e4d0135322d0c08cf6b220e"
server_key_version: "encryptionkey@0"
I20260807 11:23:23.961719 21346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:23.978084 21346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:23.982529 21346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:23.982847 21346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:23.983875 21346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:23:23.985271 21346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:23.985345 21346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:23.985497 21346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:23.985559 21346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:24.005766 21199 heartbeater.cc:503] Master 127.16.197.62:42941 was elected leader, sending a full tablet report...
I20260807 11:23:24.019887 21346 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:33859
I20260807 11:23:24.019937 21482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:33859 every 8 connection(s)
I20260807 11:23:24.021236 21346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:23:24.025904 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 21346
I20260807 11:23:24.026108 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance
I20260807 11:23:24.026546 17172 external_mini_cluster.cc:1428] Setting key c98eaec85609ef2464558d022ca9d444
Aug 07 11:23:24 dist-test-slave-2w3w krb5kdc[20954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101803, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:24.046783 21485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:24.023880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:60997 (local address 127.16.197.62:42941)
0807 11:23:24.024386 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:24.024391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:24.025447 (+ 1056us) server_negotiation.cc:407] Connection header received
0807 11:23:24.027512 (+ 2065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:24.027518 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:24.027606 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:24.027745 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:24.029267 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:24.030057 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:24.031733 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:24.032146 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:24.032314 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:24.040105 (+ 7791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:24.040110 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:24.040114 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:24.040149 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:24.043431 (+ 3282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:24.044313 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:24.044318 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:24.044320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:24.044501 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:24.044952 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:24.044957 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:24.044961 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:24.045244 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:24.045420 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:24.046139 (+ 719us) server_negotiation.cc:299] Negotiation successful
0807 11:23:24.046376 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":397,"thread_start_us":267,"threads_started":1}
I20260807 11:23:24.047777 21486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:24.024429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42941 (local address 127.16.197.3:60997)
0807 11:23:24.025269 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:24.025320 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:24.027253 (+ 1933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:24.028000 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:24.028008 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:24.028501 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:24.029070 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:24.029087 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:24.030233 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:24.030245 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:24.031579 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:24.031591 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:24.032401 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:24.032408 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:24.032654 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:24.037477 (+ 4823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:24.037503 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:24.039837 (+ 2334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:24.043580 (+ 3743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:24.043591 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:24.043616 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:24.044161 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:24.044614 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:24.044619 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:24.044623 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:24.044827 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:24.045383 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:24.045391 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:24.045757 (+ 366us) client_negotiation.cc:769] Sending connection context
0807 11:23:24.046036 (+ 279us) client_negotiation.cc:240] Negotiation successful
0807 11:23:24.046601 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":374,"threads_started":1}
I20260807 11:23:24.049196 21483 heartbeater.cc:347] Connected to a master server at 127.16.197.62:42941
I20260807 11:23:24.049535 21483 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:24.050302 21483 heartbeater.cc:511] Master 127.16.197.62:42941 requested a full tablet report, sending...
I20260807 11:23:24.051805 21001 ts_manager.cc:194] Registered new tserver with Master: 80414fdfacc64775b0039a36d8bfb17c (127.16.197.3:33859)
I20260807 11:23:24.052662 21001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.3@KRBTEST.COM'} at 127.16.197.3:60997
I20260807 11:23:24.055073 17172 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 11:23:24 dist-test-slave-2w3w krb5kdc[20954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 11:23:24 dist-test-slave-2w3w krb5kdc[20954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.1@KRBTEST.COM
I20260807 11:23:24.111575 21491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:24.095931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44286 (local address 127.16.197.1:45387)
0807 11:23:24.096625 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:24.096633 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:24.096661 (+ 28us) server_negotiation.cc:407] Connection header received
0807 11:23:24.096728 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:24.096755 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:24.097031 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:24.097161 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:24.098682 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:24.099864 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:24.101104 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:24.101711 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:24.101845 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:24.105958 (+ 4113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:24.105971 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:24.105987 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:24.106024 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:24.108927 (+ 2903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:24.109581 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:24.109588 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:24.109590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:24.109719 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:24.110076 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:24.110079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:24.110081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:24.110403 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:24.110638 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:24.110890 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 11:23:24.111170 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":364,"threads_started":1}
Aug 07 11:23:24 dist-test-slave-2w3w krb5kdc[20954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:24.130671 21485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:24.116387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59852 (local address 127.16.197.62:42941)
0807 11:23:24.116613 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:24.116619 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:24.116745 (+ 126us) server_negotiation.cc:407] Connection header received
0807 11:23:24.116958 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:24.116964 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:24.117037 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:24.117141 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:24.118412 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:24.119247 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:24.120495 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:24.120936 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:24.121069 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:24.125123 (+ 4054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:24.125128 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:24.125131 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:24.125168 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:24.127819 (+ 2651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:24.128510 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:24.128515 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:24.128519 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:24.128649 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:24.129036 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:24.129041 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:24.129044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:24.129499 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:24.129759 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:24.130090 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 11:23:24.130306 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260807 11:23:24.131412 21001 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59852
I20260807 11:23:24.132463 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 21065
I20260807 11:23:24.146066 21491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:24.133393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44290 (local address 127.16.197.1:45387)
0807 11:23:24.133565 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:24.133569 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:24.133917 (+ 348us) server_negotiation.cc:407] Connection header received
0807 11:23:24.134044 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:24.134048 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:24.134124 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:24.134206 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:24.135541 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:24.136284 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:24.137626 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:24.138233 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:24.138363 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:24.140864 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:24.140869 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:24.140872 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:24.140907 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:24.143602 (+ 2695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:24.144373 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:24.144378 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:24.144380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:24.144504 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:24.144849 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:24.144852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:24.144854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:24.145115 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:24.145297 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:24.145647 (+ 350us) server_negotiation.cc:299] Negotiation successful
0807 11:23:24.145832 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 11:23:24.146731 21193 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:24.230667 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 21065
I20260807 11:23:24.250069 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 21205
Aug 07 11:23:24 dist-test-slave-2w3w krb5kdc[20954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.2@KRBTEST.COM
I20260807 11:23:24.267062 21496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:24.251081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44206 (local address 127.16.197.2:34591)
0807 11:23:24.251666 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:24.251673 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:24.251699 (+ 26us) server_negotiation.cc:407] Connection header received
0807 11:23:24.251797 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:24.251804 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:24.252023 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:24.252150 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:24.253293 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:24.254416 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:24.255814 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:24.256347 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:24.256442 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:24.260846 (+ 4404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:24.260857 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:24.260871 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:24.260914 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:24.264182 (+ 3268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:24.264907 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:24.264915 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:24.264917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:24.265055 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:24.265506 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:24.265513 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:24.265518 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:24.265938 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:24.266188 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:24.266459 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 11:23:24.266714 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":335,"threads_started":1}
I20260807 11:23:24.268338 21335 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:24.351825 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 21205
I20260807 11:23:24.370258 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 21346
Aug 07 11:23:24 dist-test-slave-2w3w krb5kdc[20954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.3@KRBTEST.COM
I20260807 11:23:24.387128 21498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:24.371363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40004 (local address 127.16.197.3:33859)
0807 11:23:24.372096 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:24.372104 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:24.372138 (+ 34us) server_negotiation.cc:407] Connection header received
0807 11:23:24.372201 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:24.372208 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:24.372425 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:24.372555 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:24.373788 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:24.375173 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:24.376461 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:24.377148 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:24.377246 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:24.381639 (+ 4393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:24.381650 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:24.381665 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:24.381697 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:24.384521 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:24.385177 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:24.385184 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:24.385187 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:24.385328 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:24.385670 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:24.385673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:24.385675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:24.386015 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:24.386255 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:24.386520 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 11:23:24.386734 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":414,"threads_started":1}
I20260807 11:23:24.388432 21478 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:24.471635 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 21346
I20260807 11:23:24.489563 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 20968
I20260807 11:23:24.490875 21033 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:24.596948 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 20968
2026-08-07T11:23:24Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4086 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:23:24 dist-test-slave-2w3w krb5kdc[21503](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:23:24 dist-test-slave-2w3w krb5kdc[21503](info): set up 2 sockets
Aug 07 11:23:24 dist-test-slave-2w3w krb5kdc[21503](info): commencing operation
krb5kdc: starting...
W20260807 11:23:25.721341 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.006s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:23:25 dist-test-slave-2w3w krb5kdc[21503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:23:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:23:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:23:25.951414 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:43185
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:34231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:43185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:26.286046 21517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:26.286357 21517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:26.286528 21517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:26.293972 21517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:23:26.294088 21517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:26.294171 21517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:26.294289 21517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:23:26.294350 21517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:23:26.304711 21517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:34231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:43185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:43185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:26.306154 21517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:26.307555 21517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:26.319990 21523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:26.320072 21522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:26.320041 21525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:26.320847 21517 server_base.cc:1034] running on GCE node
I20260807 11:23:26.321674 21517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:26.323284 21517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:26.324569 21517 hybrid_clock.cc:648] HybridClock initialized: now 1786101806324516 us; error 39 us; skew 500 ppm
I20260807 11:23:26.325040 21517 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:26 dist-test-slave-2w3w krb5kdc[21503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101806, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:26.328498 21517 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:23:26.329717 21517 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:26.329860 21517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:26.330212 21517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:26.333501 21517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "3a152f25aad64e3ea981af739d0667b5"
format_stamp: "Formatted at 2026-08-07 11:23:26 on dist-test-slave-2w3w"
server_key: "09486d39145ccb53c9724dc3be72caca"
server_key_iv: "94287f4b26391f114e13df68cb415ad3"
server_key_version: "encryptionkey@0"
I20260807 11:23:26.334367 21517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "3a152f25aad64e3ea981af739d0667b5"
format_stamp: "Formatted at 2026-08-07 11:23:26 on dist-test-slave-2w3w"
server_key: "09486d39145ccb53c9724dc3be72caca"
server_key_iv: "94287f4b26391f114e13df68cb415ad3"
server_key_version: "encryptionkey@0"
I20260807 11:23:26.340152 21517 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260807 11:23:26.343927 21532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:26.345343 21517 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 11:23:26.345556 21517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "3a152f25aad64e3ea981af739d0667b5"
format_stamp: "Formatted at 2026-08-07 11:23:26 on dist-test-slave-2w3w"
server_key: "09486d39145ccb53c9724dc3be72caca"
server_key_iv: "94287f4b26391f114e13df68cb415ad3"
server_key_version: "encryptionkey@0"
I20260807 11:23:26.345810 21517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:26.365562 21517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:26.370165 21517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:26.370469 21517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:26.389938 21517 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:43185
I20260807 11:23:26.389964 21588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:43185 every 8 connection(s)
I20260807 11:23:26.390936 21517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:23:26.393957 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 21517
I20260807 11:23:26.394215 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:23:26.394913 17172 external_mini_cluster.cc:1428] Setting key 236247133e76e179e35867e99458e0e0
I20260807 11:23:26.394922 21589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:26.410177 21589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a152f25aad64e3ea981af739d0667b5: Bootstrap starting.
Aug 07 11:23:26 dist-test-slave-2w3w krb5kdc[21503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:26.414496 21589 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a152f25aad64e3ea981af739d0667b5: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:26.415911 21589 log.cc:824] T 00000000000000000000000000000000 P 3a152f25aad64e3ea981af739d0667b5: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:26.419430 21589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a152f25aad64e3ea981af739d0667b5: No bootstrap required, opened a new log
I20260807 11:23:26.420689 21592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:26.397025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37944 (local address 127.16.197.62:43185)
0807 11:23:26.397934 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:26.397955 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:26.398010 (+ 55us) server_negotiation.cc:407] Connection header received
0807 11:23:26.400167 (+ 2157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:26.400199 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:26.400745 (+ 546us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:26.401221 (+ 476us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:26.402638 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:26.404659 (+ 2021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:26.405985 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:26.407203 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:26.407436 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:26.411757 (+ 4321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:26.411771 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:26.411798 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:26.411865 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:26.415916 (+ 4051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:26.416824 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:26.416830 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:26.416836 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:26.417014 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:26.417484 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:26.417489 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:26.417492 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:26.418194 (+ 702us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:26.418498 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:26.418855 (+ 357us) server_negotiation.cc:299] Negotiation successful
0807 11:23:26.419315 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":375,"threads_started":1}
I20260807 11:23:26.427152 21589 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a152f25aad64e3ea981af739d0667b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a152f25aad64e3ea981af739d0667b5" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 43185 } }
I20260807 11:23:26.427479 21589 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a152f25aad64e3ea981af739d0667b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:26.427556 21589 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a152f25aad64e3ea981af739d0667b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a152f25aad64e3ea981af739d0667b5, State: Initialized, Role: FOLLOWER
I20260807 11:23:26.428082 21589 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a152f25aad64e3ea981af739d0667b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a152f25aad64e3ea981af739d0667b5" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 43185 } }
I20260807 11:23:26.428284 21589 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a152f25aad64e3ea981af739d0667b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:23:26.428381 21589 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a152f25aad64e3ea981af739d0667b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:23:26.428507 21589 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a152f25aad64e3ea981af739d0667b5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:26.431087 21589 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a152f25aad64e3ea981af739d0667b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a152f25aad64e3ea981af739d0667b5" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 43185 } }
I20260807 11:23:26.431582 21589 leader_election.cc:302] T 00000000000000000000000000000000 P 3a152f25aad64e3ea981af739d0667b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a152f25aad64e3ea981af739d0667b5; no voters:
I20260807 11:23:26.432125 21589 leader_election.cc:288] T 00000000000000000000000000000000 P 3a152f25aad64e3ea981af739d0667b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:23:26.432283 21594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a152f25aad64e3ea981af739d0667b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:26.433854 21594 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a152f25aad64e3ea981af739d0667b5 [term 1 LEADER]: Becoming Leader. State: Replica: 3a152f25aad64e3ea981af739d0667b5, State: Running, Role: LEADER
I20260807 11:23:26.434514 21589 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a152f25aad64e3ea981af739d0667b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:23:26.434350 21594 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a152f25aad64e3ea981af739d0667b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a152f25aad64e3ea981af739d0667b5" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 43185 } }
I20260807 11:23:26.437386 21596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a152f25aad64e3ea981af739d0667b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a152f25aad64e3ea981af739d0667b5. Latest consensus state: current_term: 1 leader_uuid: "3a152f25aad64e3ea981af739d0667b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a152f25aad64e3ea981af739d0667b5" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 43185 } } }
I20260807 11:23:26.437359 21595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a152f25aad64e3ea981af739d0667b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a152f25aad64e3ea981af739d0667b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a152f25aad64e3ea981af739d0667b5" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 43185 } } }
I20260807 11:23:26.437803 21596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a152f25aad64e3ea981af739d0667b5 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:26.437803 21595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a152f25aad64e3ea981af739d0667b5 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:26.438726 21600 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:23:26.444325 21600 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:23:26.453460 21600 catalog_manager.cc:1347] Generated new cluster ID: 251a4b0588d64d81a50f7ff26a63d8a7
I20260807 11:23:26.453595 21600 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:23:26.487041 21600 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:23:26.488436 21600 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:23:26.494766 21600 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a152f25aad64e3ea981af739d0667b5: Generated new TSK 0
I20260807 11:23:26.495641 21600 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.197.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
I20260807 11:23:26.555996 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:0
--local_ip_for_outbound_sockets=127.16.197.1
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:43185
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:34231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:26.893087 21616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:26.893371 21616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:26.893469 21616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:26.900529 21616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:26.900642 21616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:26.900756 21616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:23:26.912129 21616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:34231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:43185
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:26.913461 21616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:26.914820 21616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:26.928386 21622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:26.928346 21623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:26.928454 21625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:26.929255 21616 server_base.cc:1034] running on GCE node
I20260807 11:23:26.929934 21616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:26.930924 21616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:26.932196 21616 hybrid_clock.cc:648] HybridClock initialized: now 1786101806932134 us; error 54 us; skew 500 ppm
I20260807 11:23:26.932535 21616 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:26 dist-test-slave-2w3w krb5kdc[21503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101806, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:26.935895 21616 init.cc:377] Logged in from keytab as kudu/127.16.197.1@KRBTEST.COM (short username kudu)
I20260807 11:23:26.937207 21616 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:26.937340 21616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:26.937675 21616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:26.940892 21616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-0/data/instance:
uuid: "03b80be120464b4d952149fcf4a47465"
format_stamp: "Formatted at 2026-08-07 11:23:26 on dist-test-slave-2w3w"
server_key: "b204fff901dc97e78301a229033a32d8"
server_key_iv: "164343eaa54931229d7056a77e747914"
server_key_version: "encryptionkey@0"
I20260807 11:23:26.941731 21616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance:
uuid: "03b80be120464b4d952149fcf4a47465"
format_stamp: "Formatted at 2026-08-07 11:23:26 on dist-test-slave-2w3w"
server_key: "b204fff901dc97e78301a229033a32d8"
server_key_iv: "164343eaa54931229d7056a77e747914"
server_key_version: "encryptionkey@0"
I20260807 11:23:26.946820 21616 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 11:23:26.950438 21632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:26.951840 21616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:23:26.952025 21616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "03b80be120464b4d952149fcf4a47465"
format_stamp: "Formatted at 2026-08-07 11:23:26 on dist-test-slave-2w3w"
server_key: "b204fff901dc97e78301a229033a32d8"
server_key_iv: "164343eaa54931229d7056a77e747914"
server_key_version: "encryptionkey@0"
I20260807 11:23:26.952255 21616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:26.985100 21616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:26.989850 21616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:26.990193 21616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:26.991325 21616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:23:26.992774 21616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:26.992909 21616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:26.993050 21616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:26.993151 21616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:27.027552 21616 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:45077
I20260807 11:23:27.027611 21750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:45077 every 8 connection(s)
I20260807 11:23:27.028455 21616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:23:27.030059 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 21616
I20260807 11:23:27.030293 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance
I20260807 11:23:27.030941 17172 external_mini_cluster.cc:1428] Setting key 982ed5d32bf6bdcda92b8803291018f2
Aug 07 11:23:27 dist-test-slave-2w3w krb5kdc[21503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101806, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:27.055961 21754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:27.031283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:42889 (local address 127.16.197.62:43185)
0807 11:23:27.031838 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:27.031846 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:27.033025 (+ 1179us) server_negotiation.cc:407] Connection header received
0807 11:23:27.036025 (+ 3000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:27.036030 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:27.036119 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:27.036261 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:27.038476 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:27.039276 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:27.041710 (+ 2434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:27.042117 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:27.042262 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:27.047210 (+ 4948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:27.047220 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:27.047228 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:27.047277 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:27.051546 (+ 4269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:27.052654 (+ 1108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:27.052661 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:27.052664 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:27.052838 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:27.053505 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:27.053512 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:27.053515 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:27.053947 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:27.054156 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:27.055300 (+ 1144us) server_negotiation.cc:299] Negotiation successful
0807 11:23:27.055537 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":263,"threads_started":1}
I20260807 11:23:27.057192 21755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:27.031981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:43185 (local address 127.16.197.1:42889)
0807 11:23:27.032838 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:27.032900 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:27.035698 (+ 2798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:27.036543 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:27.036557 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:27.037394 (+ 837us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:27.038268 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:27.038298 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:27.039461 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:27.039469 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:27.041546 (+ 2077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:27.041559 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:27.042354 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:27.042363 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:27.042699 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:27.043467 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:27.043500 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:27.046896 (+ 3396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:27.051743 (+ 4847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:27.051756 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:27.051778 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:27.052490 (+ 712us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:27.053031 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:27.053037 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:27.053042 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:27.053293 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:27.054246 (+ 953us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:27.054255 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:27.054803 (+ 548us) client_negotiation.cc:769] Sending connection context
0807 11:23:27.055191 (+ 388us) client_negotiation.cc:240] Negotiation successful
0807 11:23:27.055854 (+ 663us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":328,"threads_started":1}
WARNING: no policy specified for kudu/127.16.197.2@KRBTEST.COM; defaulting to no policy
I20260807 11:23:27.058755 21751 heartbeater.cc:347] Connected to a master server at 127.16.197.62:43185
I20260807 11:23:27.059096 21751 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:27.059728 21751 heartbeater.cc:511] Master 127.16.197.62:43185 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.2@KRBTEST.COM" created.
I20260807 11:23:27.061756 21551 ts_manager.cc:194] Registered new tserver with Master: 03b80be120464b4d952149fcf4a47465 (127.16.197.1:45077)
I20260807 11:23:27.063303 21551 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.1@KRBTEST.COM'} at 127.16.197.1:42889
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
I20260807 11:23:27.086089 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:0
--local_ip_for_outbound_sockets=127.16.197.2
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:43185
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:34231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:27.417137 21758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:27.417416 21758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:27.417579 21758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:27.424960 21758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:27.425071 21758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:27.425257 21758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:23:27.435096 21758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:34231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:43185
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:27.436473 21758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:27.437783 21758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:27.450951 21763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:27.451144 21767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:27.451146 21764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:27.456634 21758 server_base.cc:1034] running on GCE node
I20260807 11:23:27.457360 21758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:27.458326 21758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:27.459558 21758 hybrid_clock.cc:648] HybridClock initialized: now 1786101807459513 us; error 40 us; skew 500 ppm
I20260807 11:23:27.459896 21758 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:27 dist-test-slave-2w3w krb5kdc[21503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101807, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:27.463101 21758 init.cc:377] Logged in from keytab as kudu/127.16.197.2@KRBTEST.COM (short username kudu)
I20260807 11:23:27.464413 21758 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:27.464550 21758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:27.464888 21758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:27.468086 21758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-1/data/instance:
uuid: "f179e61f50be4568b0cf6bb220cd4911"
format_stamp: "Formatted at 2026-08-07 11:23:27 on dist-test-slave-2w3w"
server_key: "48fae7d276dcf934771ca04e6329c532"
server_key_iv: "461134f340ebedeb55e35fccda0d7450"
server_key_version: "encryptionkey@0"
I20260807 11:23:27.468923 21758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance:
uuid: "f179e61f50be4568b0cf6bb220cd4911"
format_stamp: "Formatted at 2026-08-07 11:23:27 on dist-test-slave-2w3w"
server_key: "48fae7d276dcf934771ca04e6329c532"
server_key_iv: "461134f340ebedeb55e35fccda0d7450"
server_key_version: "encryptionkey@0"
I20260807 11:23:27.474050 21758 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 11:23:27.477783 21773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:27.479797 21758 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 11:23:27.480008 21758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "f179e61f50be4568b0cf6bb220cd4911"
format_stamp: "Formatted at 2026-08-07 11:23:27 on dist-test-slave-2w3w"
server_key: "48fae7d276dcf934771ca04e6329c532"
server_key_iv: "461134f340ebedeb55e35fccda0d7450"
server_key_version: "encryptionkey@0"
I20260807 11:23:27.480258 21758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:27.494869 21758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:27.498934 21758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:27.499279 21758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:27.500257 21758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:23:27.501614 21758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:27.501739 21758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:27.501878 21758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:27.501974 21758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:27.536821 21758 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:46099
I20260807 11:23:27.536880 21890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:46099 every 8 connection(s)
I20260807 11:23:27.537793 21758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:23:27.538158 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 21758
I20260807 11:23:27.538398 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance
I20260807 11:23:27.539052 17172 external_mini_cluster.cc:1428] Setting key 62d0cdf85cf6d31e5d368a644903ef18
Aug 07 11:23:27 dist-test-slave-2w3w krb5kdc[21503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101807, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:27.560963 21754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:27.540557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:45541 (local address 127.16.197.62:43185)
0807 11:23:27.540759 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:27.540765 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:27.542122 (+ 1357us) server_negotiation.cc:407] Connection header received
0807 11:23:27.544129 (+ 2007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:27.544135 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:27.544229 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:27.544343 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:27.545954 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:27.546698 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:27.549011 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:27.549550 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:27.549655 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:27.553839 (+ 4184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:27.553845 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:27.553848 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:27.553883 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:27.557096 (+ 3213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:27.558280 (+ 1184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:27.558285 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:27.558287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:27.558441 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:27.559060 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:27.559064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:27.559066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:27.559348 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:27.559490 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:27.560516 (+ 1026us) server_negotiation.cc:299] Negotiation successful
0807 11:23:27.560717 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
WARNING: no policy specified for kudu/127.16.197.3@KRBTEST.COM; defaulting to no policy
I20260807 11:23:27.562664 21893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:27.541148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:43185 (local address 127.16.197.2:45541)
0807 11:23:27.541960 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:27.542006 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:27.543853 (+ 1847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:27.544596 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:27.544618 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:27.545161 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:27.545756 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:27.545773 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:27.546873 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:27.546878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:27.548824 (+ 1946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:27.548842 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:27.549754 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:27.549759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:27.549958 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:27.550696 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:27.550731 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:27.553574 (+ 2843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:27.557274 (+ 3700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:27.557288 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:27.557334 (+ 46us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:27.558121 (+ 787us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:27.558575 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:27.558584 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:27.558590 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:27.558907 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:27.559493 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:27.559504 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:27.560060 (+ 556us) client_negotiation.cc:769] Sending connection context
0807 11:23:27.560408 (+ 348us) client_negotiation.cc:240] Negotiation successful
0807 11:23:27.561076 (+ 668us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":351,"threads_started":1}
I20260807 11:23:27.564285 21891 heartbeater.cc:347] Connected to a master server at 127.16.197.62:43185
I20260807 11:23:27.564627 21891 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:27.565344 21891 heartbeater.cc:511] Master 127.16.197.62:43185 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.3@KRBTEST.COM" created.
I20260807 11:23:27.567073 21551 ts_manager.cc:194] Registered new tserver with Master: f179e61f50be4568b0cf6bb220cd4911 (127.16.197.2:46099)
I20260807 11:23:27.567914 21551 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.2@KRBTEST.COM'} at 127.16.197.2:45541
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
I20260807 11:23:27.592654 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:0
--local_ip_for_outbound_sockets=127.16.197.3
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:43185
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:34231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:27.932446 21897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:27.932751 21897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:27.932858 21897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:27.941126 21897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:27.941250 21897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:27.941372 21897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:23:27.951745 21897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:34231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:43185
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:27.953346 21897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:27.954690 21897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:27.969070 21903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:27.968957 21906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:27.969028 21904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:27.969997 21897 server_base.cc:1034] running on GCE node
I20260807 11:23:27.970757 21897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:27.971988 21897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:27.973238 21897 hybrid_clock.cc:648] HybridClock initialized: now 1786101807973144 us; error 79 us; skew 500 ppm
I20260807 11:23:27.973647 21897 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:27 dist-test-slave-2w3w krb5kdc[21503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101807, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:27.977465 21897 init.cc:377] Logged in from keytab as kudu/127.16.197.3@KRBTEST.COM (short username kudu)
I20260807 11:23:27.978686 21897 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:27.978794 21897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:27.979151 21897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:27.982303 21897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-2/data/instance:
uuid: "b835104173f346ca93994f3f8d6f39f2"
format_stamp: "Formatted at 2026-08-07 11:23:27 on dist-test-slave-2w3w"
server_key: "c3404940a36730b364ea772398f876a1"
server_key_iv: "c15eece3b63f042c121a5b6e66df6fcb"
server_key_version: "encryptionkey@0"
I20260807 11:23:27.983213 21897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance:
uuid: "b835104173f346ca93994f3f8d6f39f2"
format_stamp: "Formatted at 2026-08-07 11:23:27 on dist-test-slave-2w3w"
server_key: "c3404940a36730b364ea772398f876a1"
server_key_iv: "c15eece3b63f042c121a5b6e66df6fcb"
server_key_version: "encryptionkey@0"
I20260807 11:23:27.988914 21897 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 11:23:27.992673 21912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:27.994123 21897 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 11:23:27.994264 21897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "b835104173f346ca93994f3f8d6f39f2"
format_stamp: "Formatted at 2026-08-07 11:23:27 on dist-test-slave-2w3w"
server_key: "c3404940a36730b364ea772398f876a1"
server_key_iv: "c15eece3b63f042c121a5b6e66df6fcb"
server_key_version: "encryptionkey@0"
I20260807 11:23:27.994463 21897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:28.014513 21897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:28.019420 21897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:28.019727 21897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:28.020798 21897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:23:28.022246 21897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:28.022372 21897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:28.022514 21897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:28.022603 21897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:28.058151 21897 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:39989
I20260807 11:23:28.058207 22030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:39989 every 8 connection(s)
I20260807 11:23:28.059103 21897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:23:28.065806 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 21897
I20260807 11:23:28.066062 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance
I20260807 11:23:28.066190 21751 heartbeater.cc:503] Master 127.16.197.62:43185 was elected leader, sending a full tablet report...
I20260807 11:23:28.066587 17172 external_mini_cluster.cc:1428] Setting key e96a636a894d1a994ec05d09b2d25c8b
Aug 07 11:23:28 dist-test-slave-2w3w krb5kdc[21503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101807, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:28.082136 22034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:28.061727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:48643 (local address 127.16.197.62:43185)
0807 11:23:28.062168 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:28.062173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:28.063559 (+ 1386us) server_negotiation.cc:407] Connection header received
0807 11:23:28.066085 (+ 2526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:28.066091 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:28.066184 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:28.066305 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:28.068626 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.069696 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.071474 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.071941 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.072047 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:28.075662 (+ 3615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:28.075667 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:28.075671 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:28.075720 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:28.078357 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:28.079251 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:28.079256 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:28.079258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:28.079431 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:28.079821 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:28.079826 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:28.079829 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:28.080106 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:28.080289 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:28.081343 (+ 1054us) server_negotiation.cc:299] Negotiation successful
0807 11:23:28.081631 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":217,"threads_started":1}
I20260807 11:23:28.083312 22035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:28.062277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:43185 (local address 127.16.197.3:48643)
0807 11:23:28.063333 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:28.063397 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:28.065720 (+ 2323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:28.066554 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:28.066569 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:28.067414 (+ 845us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:28.068255 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:28.068275 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.069873 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.069879 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:28.071326 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:28.071340 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.072169 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.072174 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:28.072380 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:28.073058 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:28.073082 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:28.075435 (+ 2353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:28.078510 (+ 3075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:28.078517 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:28.078531 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:28.079083 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:28.079531 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:28.079535 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:28.079537 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:28.079715 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:28.080269 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:28.080280 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:28.080684 (+ 404us) client_negotiation.cc:769] Sending connection context
0807 11:23:28.081203 (+ 519us) client_negotiation.cc:240] Negotiation successful
0807 11:23:28.081791 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":293,"threads_started":1}
I20260807 11:23:28.084864 22031 heartbeater.cc:347] Connected to a master server at 127.16.197.62:43185
I20260807 11:23:28.085170 22031 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:28.085861 22031 heartbeater.cc:511] Master 127.16.197.62:43185 requested a full tablet report, sending...
I20260807 11:23:28.087392 21550 ts_manager.cc:194] Registered new tserver with Master: b835104173f346ca93994f3f8d6f39f2 (127.16.197.3:39989)
I20260807 11:23:28.088267 21550 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.3@KRBTEST.COM'} at 127.16.197.3:48643
I20260807 11:23:28.095613 17172 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 11:23:28.113410 22034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:28.100660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37950 (local address 127.16.197.62:43185)
0807 11:23:28.100871 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:28.100875 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:28.101501 (+ 626us) server_negotiation.cc:407] Connection header received
0807 11:23:28.101606 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:28.101610 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:28.101681 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:28.101781 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:28.103385 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.104164 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.105397 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.105867 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.105969 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:28.108300 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:28.108305 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:28.108308 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:28.108344 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:28.110843 (+ 2499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:28.111555 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:28.111559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:28.111562 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:28.111691 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:28.112069 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:28.112072 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:28.112074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:28.112463 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:28.112668 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:28.112974 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 11:23:28.113176 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 11:23:28.117748 21550 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 11:23:28.124715 21550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:23:28.148178 22050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:28.139421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43196 (local address 127.16.197.3:39989)
0807 11:23:28.140022 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:28.140032 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:28.140067 (+ 35us) server_negotiation.cc:407] Connection header received
0807 11:23:28.140161 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:28.140167 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:28.140399 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:28.140588 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:28.141558 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.143156 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.145487 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.146602 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.146739 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:28.147022 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:28.147659 (+ 637us) server_negotiation.cc:299] Negotiation successful
0807 11:23:28.147839 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":1254,"threads_started":1}
I20260807 11:23:28.148200 22046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:28.137237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:46099 (local address 127.0.0.1:60276)
0807 11:23:28.138746 (+ 1509us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:28.138766 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:28.138895 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:28.140230 (+ 1335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:28.140234 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:28.140251 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:28.141162 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:28.141176 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.142599 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.142607 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:28.145064 (+ 2457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:28.145080 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.147005 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.147012 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:28.147251 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:28.147343 (+ 92us) client_negotiation.cc:769] Sending connection context
0807 11:23:28.147454 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 11:23:28.147700 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1358,"mutex_wait_us":323,"thread_start_us":1185,"threads_started":1}
I20260807 11:23:28.148782 22045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:28.136949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:45077 (local address 127.0.0.1:34674)
0807 11:23:28.138057 (+ 1108us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:28.138094 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:28.138296 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:28.140173 (+ 1877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:28.140182 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:28.140216 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:28.140775 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:28.140788 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.143084 (+ 2296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.143090 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:28.145513 (+ 2423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:28.145526 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.147405 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.147410 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:28.147588 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:28.147618 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 11:23:28.147675 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 11:23:28.147779 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":868,"mutex_wait_us":97,"spinlock_wait_cycles":1280,"thread_start_us":724,"threads_started":1}
I20260807 11:23:28.148780 22047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:28.137367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34674 (local address 127.16.197.1:45077)
0807 11:23:28.139483 (+ 2116us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:28.139493 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:28.139522 (+ 29us) server_negotiation.cc:407] Connection header received
0807 11:23:28.139627 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:28.139635 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:28.139908 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:28.140990 (+ 1082us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:28.141234 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.142900 (+ 1666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.145681 (+ 2781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.147160 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.147949 (+ 789us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:28.148188 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:28.148260 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 11:23:28.148434 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1937,"thread_start_us":1758,"threads_started":1}
I20260807 11:23:28.148241 22049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:28.138838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:39989 (local address 127.0.0.1:43196)
0807 11:23:28.139399 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:28.139416 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:28.139528 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:28.140896 (+ 1368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:28.140901 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:28.140936 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:28.141334 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:28.141345 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.143335 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.143346 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:28.145306 (+ 1960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:28.145322 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.147176 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.147185 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:28.147367 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:28.147404 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 11:23:28.147500 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 11:23:28.147714 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":62,"thread_start_us":741,"threads_started":1}
I20260807 11:23:28.149617 22048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:28.137908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60276 (local address 127.16.197.2:46099)
0807 11:23:28.139686 (+ 1778us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:28.139694 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:28.139718 (+ 24us) server_negotiation.cc:407] Connection header received
0807 11:23:28.139798 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:28.139816 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:28.140043 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:28.140625 (+ 582us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:28.141360 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.142439 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.145261 (+ 2822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.146725 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.148723 (+ 1998us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:28.148935 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:28.149158 (+ 223us) server_negotiation.cc:299] Negotiation successful
0807 11:23:28.149342 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":906,"thread_start_us":761,"threads_started":1}
I20260807 11:23:28.153290 21822 tablet_service.cc:1467] Processing CreateTablet for tablet 7ab46eb35d784eda9ad1688a896dee9e (DEFAULT_TABLE table=test-table [id=9ebfd856f28c45beba88f5e8442ac0f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:28.153272 21961 tablet_service.cc:1467] Processing CreateTablet for tablet 7ab46eb35d784eda9ad1688a896dee9e (DEFAULT_TABLE table=test-table [id=9ebfd856f28c45beba88f5e8442ac0f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:28.153267 21680 tablet_service.cc:1467] Processing CreateTablet for tablet 7ab46eb35d784eda9ad1688a896dee9e (DEFAULT_TABLE table=test-table [id=9ebfd856f28c45beba88f5e8442ac0f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:28.154551 21822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ab46eb35d784eda9ad1688a896dee9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:28.154647 21961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ab46eb35d784eda9ad1688a896dee9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:28.154646 21680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ab46eb35d784eda9ad1688a896dee9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:28.167271 22051 tablet_bootstrap.cc:492] T 7ab46eb35d784eda9ad1688a896dee9e P f179e61f50be4568b0cf6bb220cd4911: Bootstrap starting.
I20260807 11:23:28.171080 22052 tablet_bootstrap.cc:492] T 7ab46eb35d784eda9ad1688a896dee9e P 03b80be120464b4d952149fcf4a47465: Bootstrap starting.
I20260807 11:23:28.171133 22053 tablet_bootstrap.cc:492] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2: Bootstrap starting.
I20260807 11:23:28.171945 22051 tablet_bootstrap.cc:654] T 7ab46eb35d784eda9ad1688a896dee9e P f179e61f50be4568b0cf6bb220cd4911: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:28.173350 22051 log.cc:824] T 7ab46eb35d784eda9ad1688a896dee9e P f179e61f50be4568b0cf6bb220cd4911: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:28.174126 22052 tablet_bootstrap.cc:654] T 7ab46eb35d784eda9ad1688a896dee9e P 03b80be120464b4d952149fcf4a47465: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:28.174710 22053 tablet_bootstrap.cc:654] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:28.175421 22052 log.cc:824] T 7ab46eb35d784eda9ad1688a896dee9e P 03b80be120464b4d952149fcf4a47465: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:28.176028 22053 log.cc:824] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:28.176920 22051 tablet_bootstrap.cc:492] T 7ab46eb35d784eda9ad1688a896dee9e P f179e61f50be4568b0cf6bb220cd4911: No bootstrap required, opened a new log
I20260807 11:23:28.177189 22051 ts_tablet_manager.cc:1397] T 7ab46eb35d784eda9ad1688a896dee9e P f179e61f50be4568b0cf6bb220cd4911: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260807 11:23:28.178476 22052 tablet_bootstrap.cc:492] T 7ab46eb35d784eda9ad1688a896dee9e P 03b80be120464b4d952149fcf4a47465: No bootstrap required, opened a new log
I20260807 11:23:28.178773 22052 ts_tablet_manager.cc:1397] T 7ab46eb35d784eda9ad1688a896dee9e P 03b80be120464b4d952149fcf4a47465: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 11:23:28.179997 22053 tablet_bootstrap.cc:492] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2: No bootstrap required, opened a new log
I20260807 11:23:28.180274 22053 ts_tablet_manager.cc:1397] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 11:23:28.185047 22051 raft_consensus.cc:348] T 7ab46eb35d784eda9ad1688a896dee9e P f179e61f50be4568b0cf6bb220cd4911 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b80be120464b4d952149fcf4a47465" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45077 } } peers { permanent_uuid: "f179e61f50be4568b0cf6bb220cd4911" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46099 } } peers { permanent_uuid: "b835104173f346ca93994f3f8d6f39f2" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39989 } }
I20260807 11:23:28.185441 22051 raft_consensus.cc:374] T 7ab46eb35d784eda9ad1688a896dee9e P f179e61f50be4568b0cf6bb220cd4911 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:28.185575 22051 raft_consensus.cc:729] T 7ab46eb35d784eda9ad1688a896dee9e P f179e61f50be4568b0cf6bb220cd4911 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f179e61f50be4568b0cf6bb220cd4911, State: Initialized, Role: FOLLOWER
I20260807 11:23:28.186223 22051 consensus_queue.cc:260] T 7ab46eb35d784eda9ad1688a896dee9e P f179e61f50be4568b0cf6bb220cd4911 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b80be120464b4d952149fcf4a47465" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45077 } } peers { permanent_uuid: "f179e61f50be4568b0cf6bb220cd4911" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46099 } } peers { permanent_uuid: "b835104173f346ca93994f3f8d6f39f2" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39989 } }
I20260807 11:23:28.187390 21891 heartbeater.cc:503] Master 127.16.197.62:43185 was elected leader, sending a full tablet report...
I20260807 11:23:28.188082 22051 ts_tablet_manager.cc:1428] T 7ab46eb35d784eda9ad1688a896dee9e P f179e61f50be4568b0cf6bb220cd4911: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260807 11:23:28.189785 22052 raft_consensus.cc:348] T 7ab46eb35d784eda9ad1688a896dee9e P 03b80be120464b4d952149fcf4a47465 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b80be120464b4d952149fcf4a47465" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45077 } } peers { permanent_uuid: "f179e61f50be4568b0cf6bb220cd4911" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46099 } } peers { permanent_uuid: "b835104173f346ca93994f3f8d6f39f2" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39989 } }
I20260807 11:23:28.190253 22052 raft_consensus.cc:374] T 7ab46eb35d784eda9ad1688a896dee9e P 03b80be120464b4d952149fcf4a47465 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:28.190354 22052 raft_consensus.cc:729] T 7ab46eb35d784eda9ad1688a896dee9e P 03b80be120464b4d952149fcf4a47465 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03b80be120464b4d952149fcf4a47465, State: Initialized, Role: FOLLOWER
I20260807 11:23:28.190991 22052 consensus_queue.cc:260] T 7ab46eb35d784eda9ad1688a896dee9e P 03b80be120464b4d952149fcf4a47465 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b80be120464b4d952149fcf4a47465" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45077 } } peers { permanent_uuid: "f179e61f50be4568b0cf6bb220cd4911" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46099 } } peers { permanent_uuid: "b835104173f346ca93994f3f8d6f39f2" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39989 } }
I20260807 11:23:28.191166 22053 raft_consensus.cc:348] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b80be120464b4d952149fcf4a47465" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45077 } } peers { permanent_uuid: "f179e61f50be4568b0cf6bb220cd4911" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46099 } } peers { permanent_uuid: "b835104173f346ca93994f3f8d6f39f2" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39989 } }
I20260807 11:23:28.191648 22053 raft_consensus.cc:374] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:28.191759 22053 raft_consensus.cc:729] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b835104173f346ca93994f3f8d6f39f2, State: Initialized, Role: FOLLOWER
I20260807 11:23:28.192404 22053 consensus_queue.cc:260] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b80be120464b4d952149fcf4a47465" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45077 } } peers { permanent_uuid: "f179e61f50be4568b0cf6bb220cd4911" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46099 } } peers { permanent_uuid: "b835104173f346ca93994f3f8d6f39f2" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39989 } }
I20260807 11:23:28.193719 22052 ts_tablet_manager.cc:1428] T 7ab46eb35d784eda9ad1688a896dee9e P 03b80be120464b4d952149fcf4a47465: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260807 11:23:28.194689 22031 heartbeater.cc:503] Master 127.16.197.62:43185 was elected leader, sending a full tablet report...
I20260807 11:23:28.196038 22053 ts_tablet_manager.cc:1428] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
W20260807 11:23:28.280963 21753 tablet.cc:2369] T 7ab46eb35d784eda9ad1688a896dee9e P 03b80be120464b4d952149fcf4a47465: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 11:23:28.289806 21892 tablet.cc:2369] T 7ab46eb35d784eda9ad1688a896dee9e P f179e61f50be4568b0cf6bb220cd4911: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 11:23:28.310705 22033 tablet.cc:2369] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:23:28.446264 22059 raft_consensus.cc:482] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:23:28.446646 22059 raft_consensus.cc:504] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b80be120464b4d952149fcf4a47465" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45077 } } peers { permanent_uuid: "f179e61f50be4568b0cf6bb220cd4911" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46099 } } peers { permanent_uuid: "b835104173f346ca93994f3f8d6f39f2" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39989 } }
I20260807 11:23:28.448427 22059 leader_election.cc:288] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03b80be120464b4d952149fcf4a47465 (127.16.197.1:45077), f179e61f50be4568b0cf6bb220cd4911 (127.16.197.2:46099)
I20260807 11:23:28.454813 22035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:28.448727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:45077 (local address 127.16.197.3:39533)
0807 11:23:28.448982 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:28.449007 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:28.449154 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:28.449802 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:28.449810 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:28.449840 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:28.450107 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:28.450119 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.451349 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.451355 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:28.452765 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:28.452777 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.453997 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.454003 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:28.454207 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:28.454244 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 11:23:28.454368 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 11:23:28.454505 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":28}
I20260807 11:23:28.454849 22047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:28.449170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:39533 (local address 127.16.197.1:45077)
0807 11:23:28.449391 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:28.449398 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:28.449419 (+ 21us) server_negotiation.cc:407] Connection header received
0807 11:23:28.449501 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:28.449507 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:28.449635 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:28.449761 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:28.450266 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.451202 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.452927 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.453827 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.454003 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:28.454085 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:28.454409 (+ 324us) server_negotiation.cc:299] Negotiation successful
0807 11:23:28.454539 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 11:23:28.455087 22048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:28.449052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:55055 (local address 127.16.197.2:46099)
0807 11:23:28.449290 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:28.449296 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:28.449649 (+ 353us) server_negotiation.cc:407] Connection header received
0807 11:23:28.449828 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:28.449834 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:28.449917 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:28.450023 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:28.450552 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.451368 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.453251 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.454055 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.454242 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:28.454312 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:28.454591 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 11:23:28.454740 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 11:23:28.455231 22061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:28.448814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:46099 (local address 127.16.197.3:55055)
0807 11:23:28.449545 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:28.449559 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:28.449668 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:28.450045 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:28.450049 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:28.450076 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:28.450398 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:28.450406 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.451518 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.451523 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:28.453093 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:28.453110 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.454254 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.454260 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:28.454438 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:28.454459 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 11:23:28.454564 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 11:23:28.454754 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"mutex_wait_us":3,"thread_start_us":498,"threads_started":1}
I20260807 11:23:28.456513 21842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ab46eb35d784eda9ad1688a896dee9e" candidate_uuid: "b835104173f346ca93994f3f8d6f39f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f179e61f50be4568b0cf6bb220cd4911" is_pre_election: true
I20260807 11:23:28.456384 21702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ab46eb35d784eda9ad1688a896dee9e" candidate_uuid: "b835104173f346ca93994f3f8d6f39f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03b80be120464b4d952149fcf4a47465" is_pre_election: true
I20260807 11:23:28.457041 21842 raft_consensus.cc:2454] T 7ab46eb35d784eda9ad1688a896dee9e P f179e61f50be4568b0cf6bb220cd4911 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b835104173f346ca93994f3f8d6f39f2 in term 0.
I20260807 11:23:28.457053 21702 raft_consensus.cc:2454] T 7ab46eb35d784eda9ad1688a896dee9e P 03b80be120464b4d952149fcf4a47465 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b835104173f346ca93994f3f8d6f39f2 in term 0.
I20260807 11:23:28.457752 21913 leader_election.cc:302] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b835104173f346ca93994f3f8d6f39f2, f179e61f50be4568b0cf6bb220cd4911; no voters:
I20260807 11:23:28.458144 22059 raft_consensus.cc:2781] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:23:28.458309 22059 raft_consensus.cc:482] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:23:28.458436 22059 raft_consensus.cc:3037] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:28.460783 22059 raft_consensus.cc:504] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b80be120464b4d952149fcf4a47465" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45077 } } peers { permanent_uuid: "f179e61f50be4568b0cf6bb220cd4911" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46099 } } peers { permanent_uuid: "b835104173f346ca93994f3f8d6f39f2" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39989 } }
I20260807 11:23:28.461499 22059 leader_election.cc:288] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2 [CANDIDATE]: Term 1 election: Requested vote from peers 03b80be120464b4d952149fcf4a47465 (127.16.197.1:45077), f179e61f50be4568b0cf6bb220cd4911 (127.16.197.2:46099)
I20260807 11:23:28.461966 21702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ab46eb35d784eda9ad1688a896dee9e" candidate_uuid: "b835104173f346ca93994f3f8d6f39f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03b80be120464b4d952149fcf4a47465"
I20260807 11:23:28.462071 21842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ab46eb35d784eda9ad1688a896dee9e" candidate_uuid: "b835104173f346ca93994f3f8d6f39f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f179e61f50be4568b0cf6bb220cd4911"
I20260807 11:23:28.462215 21702 raft_consensus.cc:3037] T 7ab46eb35d784eda9ad1688a896dee9e P 03b80be120464b4d952149fcf4a47465 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:28.462285 21842 raft_consensus.cc:3037] T 7ab46eb35d784eda9ad1688a896dee9e P f179e61f50be4568b0cf6bb220cd4911 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:28.465627 21842 raft_consensus.cc:2454] T 7ab46eb35d784eda9ad1688a896dee9e P f179e61f50be4568b0cf6bb220cd4911 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b835104173f346ca93994f3f8d6f39f2 in term 1.
I20260807 11:23:28.465627 21702 raft_consensus.cc:2454] T 7ab46eb35d784eda9ad1688a896dee9e P 03b80be120464b4d952149fcf4a47465 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b835104173f346ca93994f3f8d6f39f2 in term 1.
I20260807 11:23:28.466104 21917 leader_election.cc:302] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03b80be120464b4d952149fcf4a47465, b835104173f346ca93994f3f8d6f39f2; no voters:
I20260807 11:23:28.466460 22059 raft_consensus.cc:2781] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:28.467043 22059 raft_consensus.cc:686] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2 [term 1 LEADER]: Becoming Leader. State: Replica: b835104173f346ca93994f3f8d6f39f2, State: Running, Role: LEADER
I20260807 11:23:28.467566 22059 consensus_queue.cc:237] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b80be120464b4d952149fcf4a47465" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45077 } } peers { permanent_uuid: "f179e61f50be4568b0cf6bb220cd4911" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46099 } } peers { permanent_uuid: "b835104173f346ca93994f3f8d6f39f2" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39989 } }
I20260807 11:23:28.472572 21550 catalog_manager.cc:5495] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to b835104173f346ca93994f3f8d6f39f2 (127.16.197.3). New cstate: current_term: 1 leader_uuid: "b835104173f346ca93994f3f8d6f39f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03b80be120464b4d952149fcf4a47465" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f179e61f50be4568b0cf6bb220cd4911" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b835104173f346ca93994f3f8d6f39f2" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39989 } health_report { overall_health: HEALTHY } } }
I20260807 11:23:28.517047 22050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:28.511557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54268 (local address 127.16.197.3:39989)
0807 11:23:28.511841 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:28.511848 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:28.511877 (+ 29us) server_negotiation.cc:407] Connection header received
0807 11:23:28.512067 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:28.512071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:28.512154 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:28.512287 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:23:28.512626 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.513365 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.514775 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.515292 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.515393 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:28.515948 (+ 555us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:23:28.516457 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:23:28.516584 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:28.516666 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 11:23:28.516737 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 11:23:28.525630 21702 raft_consensus.cc:1261] T 7ab46eb35d784eda9ad1688a896dee9e P 03b80be120464b4d952149fcf4a47465 [term 1 FOLLOWER]: Refusing update from remote peer b835104173f346ca93994f3f8d6f39f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:28.525630 21842 raft_consensus.cc:1261] T 7ab46eb35d784eda9ad1688a896dee9e P f179e61f50be4568b0cf6bb220cd4911 [term 1 FOLLOWER]: Refusing update from remote peer b835104173f346ca93994f3f8d6f39f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:28.526811 22063 consensus_queue.cc:1035] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f179e61f50be4568b0cf6bb220cd4911" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:28.527240 22059 consensus_queue.cc:1035] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03b80be120464b4d952149fcf4a47465" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:28.562512 21550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:23:28.562788 21550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:23:28.566432 21550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3a152f25aad64e3ea981af739d0667b5: Sending DeleteTablet for 3 replicas of tablet 7ab46eb35d784eda9ad1688a896dee9e
I20260807 11:23:28.567600 21680 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ab46eb35d784eda9ad1688a896dee9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:28 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:34674
I20260807 11:23:28.567607 21822 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ab46eb35d784eda9ad1688a896dee9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:28 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:60276
I20260807 11:23:28.568688 22075 tablet_replica.cc:330] T 7ab46eb35d784eda9ad1688a896dee9e P 03b80be120464b4d952149fcf4a47465: stopping tablet replica
I20260807 11:23:28.569132 22075 raft_consensus.cc:2229] T 7ab46eb35d784eda9ad1688a896dee9e P 03b80be120464b4d952149fcf4a47465 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:28.569615 22075 raft_consensus.cc:2258] T 7ab46eb35d784eda9ad1688a896dee9e P 03b80be120464b4d952149fcf4a47465 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:28.569736 21961 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ab46eb35d784eda9ad1688a896dee9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:28 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:43196
I20260807 11:23:28.570372 22076 tablet_replica.cc:330] T 7ab46eb35d784eda9ad1688a896dee9e P f179e61f50be4568b0cf6bb220cd4911: stopping tablet replica
I20260807 11:23:28.570683 22077 tablet_replica.cc:330] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2: stopping tablet replica
I20260807 11:23:28.570763 22076 raft_consensus.cc:2229] T 7ab46eb35d784eda9ad1688a896dee9e P f179e61f50be4568b0cf6bb220cd4911 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:28.571266 22076 raft_consensus.cc:2258] T 7ab46eb35d784eda9ad1688a896dee9e P f179e61f50be4568b0cf6bb220cd4911 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:28.571218 22077 raft_consensus.cc:2229] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2 [term 1 LEADER]: Raft consensus shutting down.
I20260807 11:23:28.571853 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 21616
I20260807 11:23:28.572258 22077 raft_consensus.cc:2258] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:28.574807 22077 ts_tablet_manager.cc:1905] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:28.576447 22076 ts_tablet_manager.cc:1905] T 7ab46eb35d784eda9ad1688a896dee9e P f179e61f50be4568b0cf6bb220cd4911: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:28.582495 22077 ts_tablet_manager.cc:1918] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:23:28.582772 22077 log.cc:1196] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-2/wal/wals/7ab46eb35d784eda9ad1688a896dee9e
I20260807 11:23:28.583483 22077 ts_tablet_manager.cc:1939] T 7ab46eb35d784eda9ad1688a896dee9e P b835104173f346ca93994f3f8d6f39f2: Deleting consensus metadata
Aug 07 11:23:28 dist-test-slave-2w3w krb5kdc[21503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.1@KRBTEST.COM
I20260807 11:23:28.585595 21534 catalog_manager.cc:4841] TS b835104173f346ca93994f3f8d6f39f2 (127.16.197.3:39989): tablet 7ab46eb35d784eda9ad1688a896dee9e (table test-table [id=9ebfd856f28c45beba88f5e8442ac0f9]) successfully deleted
I20260807 11:23:28.586115 22075 ts_tablet_manager.cc:1905] T 7ab46eb35d784eda9ad1688a896dee9e P 03b80be120464b4d952149fcf4a47465: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:28.587077 22076 ts_tablet_manager.cc:1918] T 7ab46eb35d784eda9ad1688a896dee9e P f179e61f50be4568b0cf6bb220cd4911: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:23:28.587334 22076 log.cc:1196] T 7ab46eb35d784eda9ad1688a896dee9e P f179e61f50be4568b0cf6bb220cd4911: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-1/wal/wals/7ab46eb35d784eda9ad1688a896dee9e
I20260807 11:23:28.588241 22076 ts_tablet_manager.cc:1939] T 7ab46eb35d784eda9ad1688a896dee9e P f179e61f50be4568b0cf6bb220cd4911: Deleting consensus metadata
I20260807 11:23:28.590078 21534 catalog_manager.cc:4841] TS f179e61f50be4568b0cf6bb220cd4911 (127.16.197.2:46099): tablet 7ab46eb35d784eda9ad1688a896dee9e (table test-table [id=9ebfd856f28c45beba88f5e8442ac0f9]) successfully deleted
I20260807 11:23:28.593048 22075 ts_tablet_manager.cc:1918] T 7ab46eb35d784eda9ad1688a896dee9e P 03b80be120464b4d952149fcf4a47465: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:23:28.593250 22075 log.cc:1196] T 7ab46eb35d784eda9ad1688a896dee9e P 03b80be120464b4d952149fcf4a47465: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786101765888975-17172-0/minicluster-data/ts-0/wal/wals/7ab46eb35d784eda9ad1688a896dee9e
I20260807 11:23:28.593751 22047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:28.573081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46706 (local address 127.16.197.1:45077)
0807 11:23:28.573275 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:28.573282 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:28.573705 (+ 423us) server_negotiation.cc:407] Connection header received
0807 11:23:28.574011 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:28.574017 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:28.574099 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:28.574179 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:28.576108 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.577253 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.579225 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.579867 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.579986 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:28.585563 (+ 5577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:28.585579 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:28.585596 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:28.585643 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:28.589804 (+ 4161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:28.590778 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:28.590788 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:28.590792 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:28.591045 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:28.591542 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:28.591547 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:28.591554 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:28.592087 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:28.592414 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:28.593051 (+ 637us) server_negotiation.cc:299] Negotiation successful
0807 11:23:28.593317 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 11:23:28.593791 22075 ts_tablet_manager.cc:1939] T 7ab46eb35d784eda9ad1688a896dee9e P 03b80be120464b4d952149fcf4a47465: Deleting consensus metadata
I20260807 11:23:28.595317 21746 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:28.595506 21537 catalog_manager.cc:4841] TS 03b80be120464b4d952149fcf4a47465 (127.16.197.1:45077): tablet 7ab46eb35d784eda9ad1688a896dee9e (table test-table [id=9ebfd856f28c45beba88f5e8442ac0f9]) successfully deleted
I20260807 11:23:28.687582 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 21616
I20260807 11:23:28.707392 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 21758
Aug 07 11:23:28 dist-test-slave-2w3w krb5kdc[21503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.2@KRBTEST.COM
I20260807 11:23:28.723723 22048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:28.708267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44804 (local address 127.16.197.2:46099)
0807 11:23:28.708453 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:28.708461 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:28.708484 (+ 23us) server_negotiation.cc:407] Connection header received
0807 11:23:28.708698 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:28.708704 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:28.708784 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:28.708909 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:28.710148 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.710859 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.712140 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.712560 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.712699 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:28.717534 (+ 4835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:28.717550 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:28.717567 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:28.717604 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:28.720792 (+ 3188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:28.721494 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:28.721506 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:28.721515 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:28.721649 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:28.722039 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:28.722045 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:28.722048 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:28.722451 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:28.722765 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:28.723086 (+ 321us) server_negotiation.cc:299] Negotiation successful
0807 11:23:28.723319 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 11:23:28.724655 21885 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:28.815995 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 21758
I20260807 11:23:28.842906 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 21897
Aug 07 11:23:28 dist-test-slave-2w3w krb5kdc[21503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.3@KRBTEST.COM
I20260807 11:23:28.862932 22050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:28.843808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54270 (local address 127.16.197.3:39989)
0807 11:23:28.843995 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:28.843999 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:28.844088 (+ 89us) server_negotiation.cc:407] Connection header received
0807 11:23:28.844303 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:28.844307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:28.844385 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:28.844477 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:28.846462 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.847544 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.849630 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:28.850114 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:28.850213 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:28.856468 (+ 6255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:28.856483 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:28.856499 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:28.856599 (+ 100us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:28.859820 (+ 3221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:28.860589 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:28.860605 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:28.860607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:28.860790 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:28.861223 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:28.861229 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:28.861232 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:28.861637 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:28.861981 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:28.862295 (+ 314us) server_negotiation.cc:299] Negotiation successful
0807 11:23:28.862529 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 11:23:28.863727 22026 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:28.989547 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 21897
I20260807 11:23:29.010486 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 21517
I20260807 11:23:29.011675 21584 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:29.131059 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 21517
2026-08-07T11:23:29Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4534 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T11:23:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:23:29Z Disabled control of system clock
I20260807 11:23:29.207486 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:36103
--webserver_interface=127.16.197.62
--webserver_port=0
--builtin_ntp_servers=127.16.197.20:39823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:36103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 11:23:29.553864 22088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:29.554191 22088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:29.554352 22088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:29.561781 22088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:23:29.561889 22088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:29.561928 22088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:29.561978 22088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:23:29.562072 22088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:23:29.571506 22088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:39823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:36103
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:36103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:29.572894 22088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:29.574770 22088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:29.587066 22093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:29.586994 22094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:29.587023 22096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:29.587999 22088 server_base.cc:1034] running on GCE node
I20260807 11:23:29.588773 22088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:29.590186 22088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:29.591390 22088 hybrid_clock.cc:648] HybridClock initialized: now 1786101809591341 us; error 44 us; skew 500 ppm
I20260807 11:23:29.591760 22088 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 11:23:29.594066 22088 webserver.cc:460] Webserver started at http://127.16.197.62:34431/ using document root <none> and password file <none>
I20260807 11:23:29.594796 22088 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:29.594921 22088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:29.595316 22088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:29.598346 22088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "69f914ca58f147af980873f8d58bd7e3"
format_stamp: "Formatted at 2026-08-07 11:23:29 on dist-test-slave-2w3w"
server_key: "7357b2ebb0db1d1bb480c80d980ea7b6"
server_key_iv: "c935a86f9d44df462fd90b1e96c4b756"
server_key_version: "encryptionkey@0"
I20260807 11:23:29.599251 22088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "69f914ca58f147af980873f8d58bd7e3"
format_stamp: "Formatted at 2026-08-07 11:23:29 on dist-test-slave-2w3w"
server_key: "7357b2ebb0db1d1bb480c80d980ea7b6"
server_key_iv: "c935a86f9d44df462fd90b1e96c4b756"
server_key_version: "encryptionkey@0"
I20260807 11:23:29.604511 22088 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260807 11:23:29.608104 22103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:29.609449 22088 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 11:23:29.609643 22088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "69f914ca58f147af980873f8d58bd7e3"
format_stamp: "Formatted at 2026-08-07 11:23:29 on dist-test-slave-2w3w"
server_key: "7357b2ebb0db1d1bb480c80d980ea7b6"
server_key_iv: "c935a86f9d44df462fd90b1e96c4b756"
server_key_version: "encryptionkey@0"
I20260807 11:23:29.609882 22088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:29.615869 22088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:29.616643 22088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:29.616959 22088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:29.636608 22088 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:36103
I20260807 11:23:29.636648 22156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:36103 every 8 connection(s)
I20260807 11:23:29.638019 22088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:23:29.641817 22159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:29.648276 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 22088
I20260807 11:23:29.648497 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:23:29.649123 17172 external_mini_cluster.cc:1428] Setting key 597d98c19af137319eaae227b2248d9c
I20260807 11:23:29.652722 22159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69f914ca58f147af980873f8d58bd7e3: Bootstrap starting.
I20260807 11:23:29.656268 22159 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69f914ca58f147af980873f8d58bd7e3: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:29.657481 22159 log.cc:824] T 00000000000000000000000000000000 P 69f914ca58f147af980873f8d58bd7e3: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:29.657652 22162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:29.651213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44490 (local address 127.16.197.62:36103)
0807 11:23:29.652162 (+ 949us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:29.652173 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:29.652214 (+ 41us) server_negotiation.cc:407] Connection header received
0807 11:23:29.653953 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:29.653975 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:29.654385 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:29.654803 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:29.656014 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:29.656020 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:29.656031 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:23:29.656044 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:29.656093 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:23:29.656107 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:29.656332 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:29.656487 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 11:23:29.656743 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":317,"threads_started":1}
I20260807 11:23:29.660465 22159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69f914ca58f147af980873f8d58bd7e3: No bootstrap required, opened a new log
I20260807 11:23:29.668063 22159 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69f914ca58f147af980873f8d58bd7e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f914ca58f147af980873f8d58bd7e3" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 36103 } }
I20260807 11:23:29.668435 22159 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69f914ca58f147af980873f8d58bd7e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:29.668562 22159 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69f914ca58f147af980873f8d58bd7e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69f914ca58f147af980873f8d58bd7e3, State: Initialized, Role: FOLLOWER
I20260807 11:23:29.669132 22159 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69f914ca58f147af980873f8d58bd7e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f914ca58f147af980873f8d58bd7e3" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 36103 } }
I20260807 11:23:29.669363 22159 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69f914ca58f147af980873f8d58bd7e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:23:29.669502 22159 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69f914ca58f147af980873f8d58bd7e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:23:29.669683 22159 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69f914ca58f147af980873f8d58bd7e3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:29.671888 22159 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69f914ca58f147af980873f8d58bd7e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f914ca58f147af980873f8d58bd7e3" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 36103 } }
I20260807 11:23:29.672386 22159 leader_election.cc:302] T 00000000000000000000000000000000 P 69f914ca58f147af980873f8d58bd7e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69f914ca58f147af980873f8d58bd7e3; no voters:
I20260807 11:23:29.672993 22159 leader_election.cc:288] T 00000000000000000000000000000000 P 69f914ca58f147af980873f8d58bd7e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:23:29.673141 22164 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69f914ca58f147af980873f8d58bd7e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:29.673949 22164 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69f914ca58f147af980873f8d58bd7e3 [term 1 LEADER]: Becoming Leader. State: Replica: 69f914ca58f147af980873f8d58bd7e3, State: Running, Role: LEADER
I20260807 11:23:29.674682 22164 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69f914ca58f147af980873f8d58bd7e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f914ca58f147af980873f8d58bd7e3" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 36103 } }
I20260807 11:23:29.675525 22159 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69f914ca58f147af980873f8d58bd7e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:23:29.677760 22166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69f914ca58f147af980873f8d58bd7e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69f914ca58f147af980873f8d58bd7e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f914ca58f147af980873f8d58bd7e3" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 36103 } } }
I20260807 11:23:29.677904 22168 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69f914ca58f147af980873f8d58bd7e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69f914ca58f147af980873f8d58bd7e3. Latest consensus state: current_term: 1 leader_uuid: "69f914ca58f147af980873f8d58bd7e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f914ca58f147af980873f8d58bd7e3" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 36103 } } }
I20260807 11:23:29.678220 22166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69f914ca58f147af980873f8d58bd7e3 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:29.678244 22168 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69f914ca58f147af980873f8d58bd7e3 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:29.680022 22174 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:23:29.685513 22174 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:23:29.694610 22174 catalog_manager.cc:1347] Generated new cluster ID: dc7dfd06c2654770a60ed5988ce448fd
I20260807 11:23:29.694777 22174 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:23:29.710078 22174 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:23:29.710500 22174 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:23:29.721494 22174 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69f914ca58f147af980873f8d58bd7e3: Generated new TSK 0
I20260807 11:23:29.722383 22174 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:23:29.729324 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:0
--local_ip_for_outbound_sockets=127.16.197.1
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:36103
--builtin_ntp_servers=127.16.197.20:39823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 11:23:30.066479 22185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:30.066772 22185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:30.066866 22185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:30.074250 22185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:30.074359 22185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:30.074499 22185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:23:30.084488 22185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:39823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:36103
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:30.085835 22185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:30.087656 22185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:30.100848 22194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:30.100872 22192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:30.100952 22191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:30.101747 22185 server_base.cc:1034] running on GCE node
I20260807 11:23:30.102316 22185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:30.103322 22185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:30.104516 22185 hybrid_clock.cc:648] HybridClock initialized: now 1786101810104457 us; error 55 us; skew 500 ppm
I20260807 11:23:30.104835 22185 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 11:23:30.107080 22185 webserver.cc:460] Webserver started at http://127.16.197.1:43351/ using document root <none> and password file <none>
I20260807 11:23:30.107780 22185 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:30.107896 22185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:30.108251 22185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:30.111245 22185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-0/data/instance:
uuid: "02f87f00febb4e59b2865dc593a5fd96"
format_stamp: "Formatted at 2026-08-07 11:23:30 on dist-test-slave-2w3w"
server_key: "b8ffae80f7a2a8cadb71f7e9ead3f119"
server_key_iv: "32a075d42d02d92757606af68bd9d813"
server_key_version: "encryptionkey@0"
I20260807 11:23:30.112068 22185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance:
uuid: "02f87f00febb4e59b2865dc593a5fd96"
format_stamp: "Formatted at 2026-08-07 11:23:30 on dist-test-slave-2w3w"
server_key: "b8ffae80f7a2a8cadb71f7e9ead3f119"
server_key_iv: "32a075d42d02d92757606af68bd9d813"
server_key_version: "encryptionkey@0"
I20260807 11:23:30.117553 22185 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 11:23:30.121109 22200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:30.122488 22185 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 11:23:30.122675 22185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "02f87f00febb4e59b2865dc593a5fd96"
format_stamp: "Formatted at 2026-08-07 11:23:30 on dist-test-slave-2w3w"
server_key: "b8ffae80f7a2a8cadb71f7e9ead3f119"
server_key_iv: "32a075d42d02d92757606af68bd9d813"
server_key_version: "encryptionkey@0"
I20260807 11:23:30.122864 22185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:30.128654 22185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:30.129429 22185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:30.129765 22185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:30.130765 22185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:23:30.132432 22185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:30.132520 22185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:30.132673 22185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:30.132757 22185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:30.167057 22185 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:36407
I20260807 11:23:30.167143 22319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:36407 every 8 connection(s)
I20260807 11:23:30.168448 22185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:23:30.172005 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 22185
I20260807 11:23:30.172260 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance
I20260807 11:23:30.173296 17172 external_mini_cluster.cc:1428] Setting key 92d584aadd8882e0f15bddc3c0f9db33
I20260807 11:23:30.176728 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:0
--local_ip_for_outbound_sockets=127.16.197.2
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:36103
--builtin_ntp_servers=127.16.197.20:39823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 11:23:30.178083 22322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:30.171195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:55787 (local address 127.16.197.62:36103)
0807 11:23:30.174484 (+ 3289us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:30.174490 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:30.174515 (+ 25us) server_negotiation.cc:407] Connection header received
0807 11:23:30.175624 (+ 1109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:30.175629 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:30.175692 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:30.175817 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:30.177284 (+ 1467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:30.177290 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:30.177294 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:23:30.177313 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:30.177361 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:23:30.177373 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:30.177506 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:30.177706 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 11:23:30.177782 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3160,"thread_start_us":3064,"threads_started":1}
I20260807 11:23:30.179222 22323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:30.171717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:36103 (local address 127.16.197.1:55787)
0807 11:23:30.173279 (+ 1562us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:30.173326 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:30.175322 (+ 1996us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:30.176065 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:30.176073 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:30.176165 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:30.176953 (+ 788us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 11:23:30.176956 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:30.177014 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 11:23:30.177123 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:30.177476 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:30.177488 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:30.177496 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 11:23:30.177801 (+ 305us) client_negotiation.cc:240] Negotiation successful
0807 11:23:30.178063 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1001,"thread_start_us":817,"threads_started":1}
I20260807 11:23:30.180796 22320 heartbeater.cc:347] Connected to a master server at 127.16.197.62:36103
I20260807 11:23:30.181119 22320 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:30.181735 22320 heartbeater.cc:511] Master 127.16.197.62:36103 requested a full tablet report, sending...
I20260807 11:23:30.183874 22121 ts_manager.cc:194] Registered new tserver with Master: 02f87f00febb4e59b2865dc593a5fd96 (127.16.197.1:36407)
W20260807 11:23:30.508317 22324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:30.508620 22324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:30.508807 22324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:30.516412 22324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:30.516521 22324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:30.516723 22324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:23:30.526245 22324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:39823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:36103
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:30.527791 22324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:30.529657 22324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:30.543479 22330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:30.543515 22332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:30.543428 22329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:30.544338 22324 server_base.cc:1034] running on GCE node
I20260807 11:23:30.544876 22324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:30.545773 22324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:30.547045 22324 hybrid_clock.cc:648] HybridClock initialized: now 1786101810546998 us; error 37 us; skew 500 ppm
I20260807 11:23:30.547377 22324 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 11:23:30.549686 22324 webserver.cc:460] Webserver started at http://127.16.197.2:34635/ using document root <none> and password file <none>
I20260807 11:23:30.550391 22324 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:30.550523 22324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:30.550874 22324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:30.553740 22324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-1/data/instance:
uuid: "844981cac86145689786f76c4fa9fe44"
format_stamp: "Formatted at 2026-08-07 11:23:30 on dist-test-slave-2w3w"
server_key: "9d8a728bf6d452a0ee49961ddb64bce4"
server_key_iv: "d4f0e12d2faaf08d6c4c4fc004611166"
server_key_version: "encryptionkey@0"
I20260807 11:23:30.554562 22324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance:
uuid: "844981cac86145689786f76c4fa9fe44"
format_stamp: "Formatted at 2026-08-07 11:23:30 on dist-test-slave-2w3w"
server_key: "9d8a728bf6d452a0ee49961ddb64bce4"
server_key_iv: "d4f0e12d2faaf08d6c4c4fc004611166"
server_key_version: "encryptionkey@0"
I20260807 11:23:30.559939 22324 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 11:23:30.563396 22338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:30.564713 22324 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 11:23:30.564914 22324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "844981cac86145689786f76c4fa9fe44"
format_stamp: "Formatted at 2026-08-07 11:23:30 on dist-test-slave-2w3w"
server_key: "9d8a728bf6d452a0ee49961ddb64bce4"
server_key_iv: "d4f0e12d2faaf08d6c4c4fc004611166"
server_key_version: "encryptionkey@0"
I20260807 11:23:30.565148 22324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:30.571182 22324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:30.572026 22324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:30.572319 22324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:30.573554 22324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:23:30.575358 22324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:30.575481 22324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:30.575608 22324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:30.575686 22324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:30.610945 22324 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:38241
I20260807 11:23:30.611069 22458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:38241 every 8 connection(s)
I20260807 11:23:30.612357 22324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:23:30.620198 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 22324
I20260807 11:23:30.620425 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance
I20260807 11:23:30.621064 17172 external_mini_cluster.cc:1428] Setting key b7a058a1dcfe788ac463bc37f14e96ce
I20260807 11:23:30.622120 22322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:30.615188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:54765 (local address 127.16.197.62:36103)
0807 11:23:30.615375 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:30.615380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:30.617800 (+ 2420us) server_negotiation.cc:407] Connection header received
0807 11:23:30.619666 (+ 1866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:30.619678 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:30.619776 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:30.619897 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:30.621364 (+ 1467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:30.621370 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:30.621374 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:23:30.621395 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:30.621436 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:23:30.621453 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:30.621590 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:30.621770 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 11:23:30.621843 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 11:23:30.623183 22461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:30.615758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:36103 (local address 127.16.197.2:54765)
0807 11:23:30.617607 (+ 1849us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:30.617652 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:30.619391 (+ 1739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:30.620108 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:30.620116 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:30.620206 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:30.621019 (+ 813us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 11:23:30.621021 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:30.621058 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 11:23:30.621143 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:30.621565 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:30.621572 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:30.621578 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 11:23:30.621838 (+ 260us) client_negotiation.cc:240] Negotiation successful
0807 11:23:30.622110 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1393,"thread_start_us":1165,"threads_started":1}
I20260807 11:23:30.624310 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:0
--local_ip_for_outbound_sockets=127.16.197.3
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:36103
--builtin_ntp_servers=127.16.197.20:39823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 11:23:30.624677 22459 heartbeater.cc:347] Connected to a master server at 127.16.197.62:36103
I20260807 11:23:30.624984 22459 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:30.625543 22459 heartbeater.cc:511] Master 127.16.197.62:36103 requested a full tablet report, sending...
I20260807 11:23:30.627017 22121 ts_manager.cc:194] Registered new tserver with Master: 844981cac86145689786f76c4fa9fe44 (127.16.197.2:38241)
W20260807 11:23:30.961153 22462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:30.961462 22462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:30.961632 22462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:30.969118 22462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:30.969228 22462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:30.969427 22462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:23:30.978938 22462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:39823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:36103
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:30.980363 22462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:30.982239 22462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:30.996080 22470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:30.996124 22467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:30.996022 22468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:30.996985 22462 server_base.cc:1034] running on GCE node
I20260807 11:23:30.997658 22462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:30.998680 22462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:30.999912 22462 hybrid_clock.cc:648] HybridClock initialized: now 1786101810999830 us; error 75 us; skew 500 ppm
I20260807 11:23:31.000296 22462 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 11:23:31.002727 22462 webserver.cc:460] Webserver started at http://127.16.197.3:35941/ using document root <none> and password file <none>
I20260807 11:23:31.003572 22462 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:31.003707 22462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:31.004063 22462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:31.007272 22462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-2/data/instance:
uuid: "bfae403357304076a248f662f6666a8b"
format_stamp: "Formatted at 2026-08-07 11:23:31 on dist-test-slave-2w3w"
server_key: "8417fad0dfd934c170f225685730e3d4"
server_key_iv: "123694be636838caf3df7bddcd449577"
server_key_version: "encryptionkey@0"
I20260807 11:23:31.008152 22462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance:
uuid: "bfae403357304076a248f662f6666a8b"
format_stamp: "Formatted at 2026-08-07 11:23:31 on dist-test-slave-2w3w"
server_key: "8417fad0dfd934c170f225685730e3d4"
server_key_iv: "123694be636838caf3df7bddcd449577"
server_key_version: "encryptionkey@0"
I20260807 11:23:31.013979 22462 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 11:23:31.017791 22476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:31.019234 22462 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:23:31.019429 22462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "bfae403357304076a248f662f6666a8b"
format_stamp: "Formatted at 2026-08-07 11:23:31 on dist-test-slave-2w3w"
server_key: "8417fad0dfd934c170f225685730e3d4"
server_key_iv: "123694be636838caf3df7bddcd449577"
server_key_version: "encryptionkey@0"
I20260807 11:23:31.019668 22462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:31.025576 22462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:31.026429 22462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:31.026760 22462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:31.027819 22462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:23:31.029238 22462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:31.029367 22462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:31.029511 22462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:31.029603 22462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:31.066777 22462 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:39187
I20260807 11:23:31.066812 22595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:39187 every 8 connection(s)
I20260807 11:23:31.068149 22462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:23:31.076576 22322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:31.070978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:57733 (local address 127.16.197.62:36103)
0807 11:23:31.071207 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:31.071214 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:31.072623 (+ 1409us) server_negotiation.cc:407] Connection header received
0807 11:23:31.074289 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:31.074293 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:31.074351 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:31.074448 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:31.075857 (+ 1409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:31.075861 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:31.075865 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:23:31.075881 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:31.075915 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:23:31.075926 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:31.076025 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:31.076196 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 11:23:31.076250 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 11:23:31.077322 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 22462
I20260807 11:23:31.077505 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance
I20260807 11:23:31.077885 22598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:31.071547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:36103 (local address 127.16.197.3:57733)
0807 11:23:31.072461 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:31.072504 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:31.073995 (+ 1491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:31.074655 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:31.074664 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:31.074767 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:31.075562 (+ 795us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 11:23:31.075565 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:31.075602 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 11:23:31.075687 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:31.076019 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:31.076026 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:31.076032 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 11:23:31.076245 (+ 213us) client_negotiation.cc:240] Negotiation successful
0807 11:23:31.076564 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":282,"threads_started":1}
I20260807 11:23:31.078099 17172 external_mini_cluster.cc:1428] Setting key ae3dd0faf5f31eeb5ad80f427d1ac9fe
I20260807 11:23:31.079488 22596 heartbeater.cc:347] Connected to a master server at 127.16.197.62:36103
I20260807 11:23:31.079845 22596 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:31.080587 22596 heartbeater.cc:511] Master 127.16.197.62:36103 requested a full tablet report, sending...
I20260807 11:23:31.081993 22121 ts_manager.cc:194] Registered new tserver with Master: bfae403357304076a248f662f6666a8b (127.16.197.3:39187)
I20260807 11:23:31.093379 17172 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 11:23:31.100476 22322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:31.098027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44504 (local address 127.16.197.62:36103)
0807 11:23:31.098195 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:31.098200 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:31.098686 (+ 486us) server_negotiation.cc:407] Connection header received
0807 11:23:31.098831 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:31.098836 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:31.098890 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:31.099044 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:31.099937 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:31.099941 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:31.099945 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:23:31.099962 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:31.099997 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:23:31.100009 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:31.100121 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:31.100199 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 11:23:31.100261 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 11:23:31.104532 22121 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44504:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 11:23:31.111363 22121 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:23:31.141027 22610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:31.123470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:38241 (local address 127.0.0.1:58510)
0807 11:23:31.134551 (+ 11081us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:31.134618 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:31.134863 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:31.138319 (+ 3456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:31.138328 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:31.138370 (+ 42us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:31.139329 (+ 959us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 11:23:31.139332 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:31.139369 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 11:23:31.139482 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:31.140241 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:31.140253 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:31.140260 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 11:23:31.140451 (+ 191us) client_negotiation.cc:240] Negotiation successful
0807 11:23:31.140565 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10637,"thread_start_us":11558,"threads_started":1}
I20260807 11:23:31.141322 22609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:31.123494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58510 (local address 127.16.197.2:38241)
0807 11:23:31.137177 (+ 13683us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:31.137187 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:31.137225 (+ 38us) server_negotiation.cc:407] Connection header received
0807 11:23:31.137320 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:31.137330 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:31.137621 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:31.137800 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:31.139743 (+ 1943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:31.139757 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:31.139777 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:23:31.139796 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:31.139859 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:23:31.139883 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:31.140830 (+ 947us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:31.140897 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 11:23:31.141018 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13528,"thread_start_us":13405,"threads_started":1}
I20260807 11:23:31.141579 22612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:31.124247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45520 (local address 127.16.197.3:39187)
0807 11:23:31.137640 (+ 13393us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:31.137648 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:31.137686 (+ 38us) server_negotiation.cc:407] Connection header received
0807 11:23:31.137765 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:31.137773 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:31.137971 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:31.138882 (+ 911us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:31.140066 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:31.140075 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:31.140092 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:23:31.140112 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:31.140164 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:23:31.140186 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:31.140677 (+ 491us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:31.140747 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 11:23:31.140850 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13225,"thread_start_us":13046,"threads_started":1}
I20260807 11:23:31.141947 22611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:31.123622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:39187 (local address 127.0.0.1:45520)
0807 11:23:31.136464 (+ 12842us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:31.136491 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:31.136630 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:31.138197 (+ 1567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:31.138205 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:31.138311 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:31.139446 (+ 1135us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 11:23:31.139452 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:31.139473 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 11:23:31.139530 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:31.140323 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:31.140328 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:31.140333 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 11:23:31.140464 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 11:23:31.140587 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12627,"mutex_wait_us":51,"thread_start_us":12428,"threads_started":1}
I20260807 11:23:31.143724 22614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:31.139312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:36407 (local address 127.0.0.1:45172)
0807 11:23:31.140042 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:31.140062 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:31.140154 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:31.141629 (+ 1475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:31.141637 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:31.141670 (+ 33us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:31.142432 (+ 762us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 11:23:31.142436 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:31.142452 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 11:23:31.142496 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:31.143110 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:31.143115 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:31.143124 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 11:23:31.143273 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 11:23:31.143325 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":447,"threads_started":1}
I20260807 11:23:31.144577 22613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:31.137705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45172 (local address 127.16.197.1:36407)
0807 11:23:31.140100 (+ 2395us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:31.140120 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:31.140147 (+ 27us) server_negotiation.cc:407] Connection header received
0807 11:23:31.140327 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:31.140336 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:31.140540 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:31.140686 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:31.142670 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:31.142677 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:31.142689 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:23:31.142719 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:31.142782 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:23:31.142802 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:31.142945 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:31.144151 (+ 1206us) server_negotiation.cc:299] Negotiation successful
0807 11:23:31.144249 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1436,"thread_start_us":1245,"threads_started":1}
I20260807 11:23:31.145610 22388 tablet_service.cc:1467] Processing CreateTablet for tablet a7851add5897418d9026e185cbe9db73 (DEFAULT_TABLE table=test-table [id=9146e4fed13f4d309f60336fbd576778]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:31.145586 22528 tablet_service.cc:1467] Processing CreateTablet for tablet a7851add5897418d9026e185cbe9db73 (DEFAULT_TABLE table=test-table [id=9146e4fed13f4d309f60336fbd576778]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:31.147064 22388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7851add5897418d9026e185cbe9db73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:31.147073 22528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7851add5897418d9026e185cbe9db73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:31.148236 22251 tablet_service.cc:1467] Processing CreateTablet for tablet a7851add5897418d9026e185cbe9db73 (DEFAULT_TABLE table=test-table [id=9146e4fed13f4d309f60336fbd576778]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:31.149600 22251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7851add5897418d9026e185cbe9db73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:31.159098 22615 tablet_bootstrap.cc:492] T a7851add5897418d9026e185cbe9db73 P 844981cac86145689786f76c4fa9fe44: Bootstrap starting.
I20260807 11:23:31.161721 22616 tablet_bootstrap.cc:492] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b: Bootstrap starting.
I20260807 11:23:31.164184 22615 tablet_bootstrap.cc:654] T a7851add5897418d9026e185cbe9db73 P 844981cac86145689786f76c4fa9fe44: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:31.164359 22617 tablet_bootstrap.cc:492] T a7851add5897418d9026e185cbe9db73 P 02f87f00febb4e59b2865dc593a5fd96: Bootstrap starting.
I20260807 11:23:31.165371 22616 tablet_bootstrap.cc:654] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:31.165710 22615 log.cc:824] T a7851add5897418d9026e185cbe9db73 P 844981cac86145689786f76c4fa9fe44: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:31.166785 22616 log.cc:824] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:31.167726 22617 tablet_bootstrap.cc:654] T a7851add5897418d9026e185cbe9db73 P 02f87f00febb4e59b2865dc593a5fd96: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:31.169062 22617 log.cc:824] T a7851add5897418d9026e185cbe9db73 P 02f87f00febb4e59b2865dc593a5fd96: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:31.173906 22616 tablet_bootstrap.cc:492] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b: No bootstrap required, opened a new log
I20260807 11:23:31.174257 22616 ts_tablet_manager.cc:1397] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260807 11:23:31.182271 22615 tablet_bootstrap.cc:492] T a7851add5897418d9026e185cbe9db73 P 844981cac86145689786f76c4fa9fe44: No bootstrap required, opened a new log
I20260807 11:23:31.182276 22617 tablet_bootstrap.cc:492] T a7851add5897418d9026e185cbe9db73 P 02f87f00febb4e59b2865dc593a5fd96: No bootstrap required, opened a new log
I20260807 11:23:31.182572 22615 ts_tablet_manager.cc:1397] T a7851add5897418d9026e185cbe9db73 P 844981cac86145689786f76c4fa9fe44: Time spent bootstrapping tablet: real 0.024s user 0.007s sys 0.010s
I20260807 11:23:31.182585 22617 ts_tablet_manager.cc:1397] T a7851add5897418d9026e185cbe9db73 P 02f87f00febb4e59b2865dc593a5fd96: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.008s
I20260807 11:23:31.186857 22320 heartbeater.cc:503] Master 127.16.197.62:36103 was elected leader, sending a full tablet report...
I20260807 11:23:31.186901 22616 raft_consensus.cc:348] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "844981cac86145689786f76c4fa9fe44" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38241 } } peers { permanent_uuid: "bfae403357304076a248f662f6666a8b" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39187 } } peers { permanent_uuid: "02f87f00febb4e59b2865dc593a5fd96" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 36407 } }
I20260807 11:23:31.187386 22616 raft_consensus.cc:374] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:31.187477 22616 raft_consensus.cc:729] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfae403357304076a248f662f6666a8b, State: Initialized, Role: FOLLOWER
I20260807 11:23:31.188215 22616 consensus_queue.cc:260] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "844981cac86145689786f76c4fa9fe44" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38241 } } peers { permanent_uuid: "bfae403357304076a248f662f6666a8b" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39187 } } peers { permanent_uuid: "02f87f00febb4e59b2865dc593a5fd96" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 36407 } }
I20260807 11:23:31.191797 22615 raft_consensus.cc:348] T a7851add5897418d9026e185cbe9db73 P 844981cac86145689786f76c4fa9fe44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "844981cac86145689786f76c4fa9fe44" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38241 } } peers { permanent_uuid: "bfae403357304076a248f662f6666a8b" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39187 } } peers { permanent_uuid: "02f87f00febb4e59b2865dc593a5fd96" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 36407 } }
I20260807 11:23:31.192252 22615 raft_consensus.cc:374] T a7851add5897418d9026e185cbe9db73 P 844981cac86145689786f76c4fa9fe44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:31.192368 22615 raft_consensus.cc:729] T a7851add5897418d9026e185cbe9db73 P 844981cac86145689786f76c4fa9fe44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 844981cac86145689786f76c4fa9fe44, State: Initialized, Role: FOLLOWER
I20260807 11:23:31.192965 22596 heartbeater.cc:503] Master 127.16.197.62:36103 was elected leader, sending a full tablet report...
I20260807 11:23:31.192992 22615 consensus_queue.cc:260] T a7851add5897418d9026e185cbe9db73 P 844981cac86145689786f76c4fa9fe44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "844981cac86145689786f76c4fa9fe44" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38241 } } peers { permanent_uuid: "bfae403357304076a248f662f6666a8b" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39187 } } peers { permanent_uuid: "02f87f00febb4e59b2865dc593a5fd96" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 36407 } }
I20260807 11:23:31.193620 22616 ts_tablet_manager.cc:1428] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b: Time spent starting tablet: real 0.019s user 0.016s sys 0.004s
I20260807 11:23:31.194257 22459 heartbeater.cc:503] Master 127.16.197.62:36103 was elected leader, sending a full tablet report...
I20260807 11:23:31.194824 22615 ts_tablet_manager.cc:1428] T a7851add5897418d9026e185cbe9db73 P 844981cac86145689786f76c4fa9fe44: Time spent starting tablet: real 0.012s user 0.012s sys 0.001s
I20260807 11:23:31.197320 22617 raft_consensus.cc:348] T a7851add5897418d9026e185cbe9db73 P 02f87f00febb4e59b2865dc593a5fd96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "844981cac86145689786f76c4fa9fe44" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38241 } } peers { permanent_uuid: "bfae403357304076a248f662f6666a8b" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39187 } } peers { permanent_uuid: "02f87f00febb4e59b2865dc593a5fd96" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 36407 } }
I20260807 11:23:31.197793 22617 raft_consensus.cc:374] T a7851add5897418d9026e185cbe9db73 P 02f87f00febb4e59b2865dc593a5fd96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:31.197916 22617 raft_consensus.cc:729] T a7851add5897418d9026e185cbe9db73 P 02f87f00febb4e59b2865dc593a5fd96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02f87f00febb4e59b2865dc593a5fd96, State: Initialized, Role: FOLLOWER
I20260807 11:23:31.198443 22617 consensus_queue.cc:260] T a7851add5897418d9026e185cbe9db73 P 02f87f00febb4e59b2865dc593a5fd96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "844981cac86145689786f76c4fa9fe44" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38241 } } peers { permanent_uuid: "bfae403357304076a248f662f6666a8b" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39187 } } peers { permanent_uuid: "02f87f00febb4e59b2865dc593a5fd96" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 36407 } }
I20260807 11:23:31.200196 22617 ts_tablet_manager.cc:1428] T a7851add5897418d9026e185cbe9db73 P 02f87f00febb4e59b2865dc593a5fd96: Time spent starting tablet: real 0.018s user 0.015s sys 0.000s
W20260807 11:23:31.320319 22597 tablet.cc:2369] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 11:23:31.364630 22460 tablet.cc:2369] T a7851add5897418d9026e185cbe9db73 P 844981cac86145689786f76c4fa9fe44: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 11:23:31.421027 22321 tablet.cc:2369] T a7851add5897418d9026e185cbe9db73 P 02f87f00febb4e59b2865dc593a5fd96: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:23:31.477162 22621 raft_consensus.cc:482] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:23:31.477535 22621 raft_consensus.cc:504] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "844981cac86145689786f76c4fa9fe44" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38241 } } peers { permanent_uuid: "bfae403357304076a248f662f6666a8b" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39187 } } peers { permanent_uuid: "02f87f00febb4e59b2865dc593a5fd96" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 36407 } }
I20260807 11:23:31.479396 22621 leader_election.cc:288] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 844981cac86145689786f76c4fa9fe44 (127.16.197.2:38241), 02f87f00febb4e59b2865dc593a5fd96 (127.16.197.1:36407)
I20260807 11:23:31.482249 22609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:31.479877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:56427 (local address 127.16.197.2:38241)
0807 11:23:31.480077 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:31.480086 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:31.480274 (+ 188us) server_negotiation.cc:407] Connection header received
0807 11:23:31.480497 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:31.480505 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:31.480571 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:31.480681 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:31.481648 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:31.481653 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:31.481658 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:23:31.481681 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:31.481710 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:23:31.481721 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:31.481846 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:31.481930 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 11:23:31.481991 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 11:23:31.482249 22598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:31.479882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:38241 (local address 127.16.197.3:56427)
0807 11:23:31.480181 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:31.480201 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:31.480351 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:31.480717 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:31.480728 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:31.480754 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:31.481437 (+ 683us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 11:23:31.481439 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:31.481453 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 11:23:31.481494 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:31.481822 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:31.481827 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:31.481832 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 11:23:31.481943 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 11:23:31.481992 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185}
I20260807 11:23:31.482767 22613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:31.479959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:51517 (local address 127.16.197.1:36407)
0807 11:23:31.480163 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:31.480171 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:31.480861 (+ 690us) server_negotiation.cc:407] Connection header received
0807 11:23:31.481066 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:31.481070 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:31.481118 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:31.481227 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:31.482172 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:31.482177 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:31.482181 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:23:31.482206 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:31.482236 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:23:31.482247 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:31.482369 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:31.482507 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 11:23:31.482571 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 11:23:31.483036 22624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:31.479959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:36407 (local address 127.16.197.3:51517)
0807 11:23:31.480739 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:31.480754 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:31.480916 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:31.481247 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:31.481252 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:31.481276 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:31.481967 (+ 691us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 11:23:31.481970 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:31.481982 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 11:23:31.482018 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:31.482345 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:31.482349 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:31.482354 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 11:23:31.482506 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 11:23:31.482560 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":430,"threads_started":1}
I20260807 11:23:31.483461 22413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7851add5897418d9026e185cbe9db73" candidate_uuid: "bfae403357304076a248f662f6666a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "844981cac86145689786f76c4fa9fe44" is_pre_election: true
I20260807 11:23:31.484050 22413 raft_consensus.cc:2454] T a7851add5897418d9026e185cbe9db73 P 844981cac86145689786f76c4fa9fe44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfae403357304076a248f662f6666a8b in term 0.
I20260807 11:23:31.484041 22272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7851add5897418d9026e185cbe9db73" candidate_uuid: "bfae403357304076a248f662f6666a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02f87f00febb4e59b2865dc593a5fd96" is_pre_election: true
I20260807 11:23:31.484603 22272 raft_consensus.cc:2454] T a7851add5897418d9026e185cbe9db73 P 02f87f00febb4e59b2865dc593a5fd96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfae403357304076a248f662f6666a8b in term 0.
I20260807 11:23:31.484836 22480 leader_election.cc:302] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 844981cac86145689786f76c4fa9fe44, bfae403357304076a248f662f6666a8b; no voters:
I20260807 11:23:31.485245 22621 raft_consensus.cc:2781] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:23:31.485394 22621 raft_consensus.cc:482] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:23:31.485496 22621 raft_consensus.cc:3037] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:31.487830 22621 raft_consensus.cc:504] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "844981cac86145689786f76c4fa9fe44" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38241 } } peers { permanent_uuid: "bfae403357304076a248f662f6666a8b" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39187 } } peers { permanent_uuid: "02f87f00febb4e59b2865dc593a5fd96" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 36407 } }
I20260807 11:23:31.488435 22621 leader_election.cc:288] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b [CANDIDATE]: Term 1 election: Requested vote from peers 844981cac86145689786f76c4fa9fe44 (127.16.197.2:38241), 02f87f00febb4e59b2865dc593a5fd96 (127.16.197.1:36407)
I20260807 11:23:31.488991 22413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7851add5897418d9026e185cbe9db73" candidate_uuid: "bfae403357304076a248f662f6666a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "844981cac86145689786f76c4fa9fe44"
I20260807 11:23:31.488993 22272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7851add5897418d9026e185cbe9db73" candidate_uuid: "bfae403357304076a248f662f6666a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02f87f00febb4e59b2865dc593a5fd96"
I20260807 11:23:31.489210 22272 raft_consensus.cc:3037] T a7851add5897418d9026e185cbe9db73 P 02f87f00febb4e59b2865dc593a5fd96 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:31.489210 22413 raft_consensus.cc:3037] T a7851add5897418d9026e185cbe9db73 P 844981cac86145689786f76c4fa9fe44 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:31.492584 22272 raft_consensus.cc:2454] T a7851add5897418d9026e185cbe9db73 P 02f87f00febb4e59b2865dc593a5fd96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfae403357304076a248f662f6666a8b in term 1.
I20260807 11:23:31.492580 22413 raft_consensus.cc:2454] T a7851add5897418d9026e185cbe9db73 P 844981cac86145689786f76c4fa9fe44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfae403357304076a248f662f6666a8b in term 1.
I20260807 11:23:31.493064 22480 leader_election.cc:302] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 844981cac86145689786f76c4fa9fe44, bfae403357304076a248f662f6666a8b; no voters:
I20260807 11:23:31.493386 22621 raft_consensus.cc:2781] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:31.493852 22621 raft_consensus.cc:686] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b [term 1 LEADER]: Becoming Leader. State: Replica: bfae403357304076a248f662f6666a8b, State: Running, Role: LEADER
I20260807 11:23:31.494347 22621 consensus_queue.cc:237] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "844981cac86145689786f76c4fa9fe44" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38241 } } peers { permanent_uuid: "bfae403357304076a248f662f6666a8b" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39187 } } peers { permanent_uuid: "02f87f00febb4e59b2865dc593a5fd96" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 36407 } }
I20260807 11:23:31.499080 22121 catalog_manager.cc:5495] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b reported cstate change: term changed from 0 to 1, leader changed from <none> to bfae403357304076a248f662f6666a8b (127.16.197.3). New cstate: current_term: 1 leader_uuid: "bfae403357304076a248f662f6666a8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "844981cac86145689786f76c4fa9fe44" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfae403357304076a248f662f6666a8b" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02f87f00febb4e59b2865dc593a5fd96" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 36407 } health_report { overall_health: UNKNOWN } } }
I20260807 11:23:31.515108 22612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:31.512514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45532 (local address 127.16.197.3:39187)
0807 11:23:31.512741 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:31.512748 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:31.512887 (+ 139us) server_negotiation.cc:407] Connection header received
0807 11:23:31.513138 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:31.513142 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:31.513220 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:31.513355 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:31.514397 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:31.514408 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:31.514411 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:23:31.514426 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:31.514457 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:23:31.514469 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:31.514602 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:31.514700 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 11:23:31.514810 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 11:23:31.524009 22413 raft_consensus.cc:1261] T a7851add5897418d9026e185cbe9db73 P 844981cac86145689786f76c4fa9fe44 [term 1 FOLLOWER]: Refusing update from remote peer bfae403357304076a248f662f6666a8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:31.524088 22272 raft_consensus.cc:1261] T a7851add5897418d9026e185cbe9db73 P 02f87f00febb4e59b2865dc593a5fd96 [term 1 FOLLOWER]: Refusing update from remote peer bfae403357304076a248f662f6666a8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:31.525120 22621 consensus_queue.cc:1035] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "02f87f00febb4e59b2865dc593a5fd96" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 36407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:31.525496 22625 consensus_queue.cc:1035] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "844981cac86145689786f76c4fa9fe44" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:31.537760 22629 mvcc.cc:205] Tried to move back new op lower bound from 7315873019990401024 to 7315873019884879872. Current Snapshot: MvccSnapshot[applied={T|T < 7315873019990401024}]
I20260807 11:23:31.542920 22630 mvcc.cc:205] Tried to move back new op lower bound from 7315873019990401024 to 7315873019884879872. Current Snapshot: MvccSnapshot[applied={T|T < 7315873019990401024}]
I20260807 11:23:31.551616 22120 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44504:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:23:31.551862 22120 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44504:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:23:31.555784 22120 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 69f914ca58f147af980873f8d58bd7e3: Sending DeleteTablet for 3 replicas of tablet a7851add5897418d9026e185cbe9db73
I20260807 11:23:31.556936 22388 tablet_service.cc:1514] Processing DeleteTablet for tablet a7851add5897418d9026e185cbe9db73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:31 UTC) from {username='slave'} at 127.0.0.1:58510
I20260807 11:23:31.557132 22251 tablet_service.cc:1514] Processing DeleteTablet for tablet a7851add5897418d9026e185cbe9db73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:31 UTC) from {username='slave'} at 127.0.0.1:45172
I20260807 11:23:31.557355 22528 tablet_service.cc:1514] Processing DeleteTablet for tablet a7851add5897418d9026e185cbe9db73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:31 UTC) from {username='slave'} at 127.0.0.1:45520
I20260807 11:23:31.559067 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 22185
I20260807 11:23:31.572747 22638 tablet_replica.cc:330] T a7851add5897418d9026e185cbe9db73 P 844981cac86145689786f76c4fa9fe44: stopping tablet replica
I20260807 11:23:31.574219 22638 raft_consensus.cc:2229] T a7851add5897418d9026e185cbe9db73 P 844981cac86145689786f76c4fa9fe44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:31.575001 22638 raft_consensus.cc:2258] T a7851add5897418d9026e185cbe9db73 P 844981cac86145689786f76c4fa9fe44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:31.576103 22640 tablet_replica.cc:330] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b: stopping tablet replica
I20260807 11:23:31.576742 22639 tablet_replica.cc:330] T a7851add5897418d9026e185cbe9db73 P 02f87f00febb4e59b2865dc593a5fd96: stopping tablet replica
I20260807 11:23:31.577270 22640 raft_consensus.cc:2229] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b [term 1 LEADER]: Raft consensus shutting down.
I20260807 11:23:31.577158 22613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:31.562489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45186 (local address 127.16.197.1:36407)
0807 11:23:31.562725 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:31.562734 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:31.573476 (+ 10742us) server_negotiation.cc:407] Connection header received
0807 11:23:31.573697 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:31.573706 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:31.573872 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:31.574023 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:31.576228 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:31.576243 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:31.576250 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:23:31.576283 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:31.576338 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:23:31.576355 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:31.576471 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:31.576745 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 11:23:31.576817 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 11:23:31.577701 22638 ts_tablet_manager.cc:1905] T a7851add5897418d9026e185cbe9db73 P 844981cac86145689786f76c4fa9fe44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:31.578230 22315 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:31.579276 22640 raft_consensus.cc:2258] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:31.581281 22640 ts_tablet_manager.cc:1905] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:31.585384 22638 ts_tablet_manager.cc:1918] T a7851add5897418d9026e185cbe9db73 P 844981cac86145689786f76c4fa9fe44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:23:31.585567 22638 log.cc:1196] T a7851add5897418d9026e185cbe9db73 P 844981cac86145689786f76c4fa9fe44: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-1/wal/wals/a7851add5897418d9026e185cbe9db73
I20260807 11:23:31.586174 22638 ts_tablet_manager.cc:1939] T a7851add5897418d9026e185cbe9db73 P 844981cac86145689786f76c4fa9fe44: Deleting consensus metadata
I20260807 11:23:31.588230 22107 catalog_manager.cc:4841] TS 844981cac86145689786f76c4fa9fe44 (127.16.197.2:38241): tablet a7851add5897418d9026e185cbe9db73 (table test-table [id=9146e4fed13f4d309f60336fbd576778]) successfully deleted
I20260807 11:23:31.590050 22640 ts_tablet_manager.cc:1918] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:23:31.590323 22640 log.cc:1196] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786101765888975-17172-0/minicluster-data/ts-2/wal/wals/a7851add5897418d9026e185cbe9db73
I20260807 11:23:31.591043 22640 ts_tablet_manager.cc:1939] T a7851add5897418d9026e185cbe9db73 P bfae403357304076a248f662f6666a8b: Deleting consensus metadata
I20260807 11:23:31.593024 22104 catalog_manager.cc:4841] TS bfae403357304076a248f662f6666a8b (127.16.197.3:39187): tablet a7851add5897418d9026e185cbe9db73 (table test-table [id=9146e4fed13f4d309f60336fbd576778]) successfully deleted
I20260807 11:23:31.578563 22639 raft_consensus.cc:2229] T a7851add5897418d9026e185cbe9db73 P 02f87f00febb4e59b2865dc593a5fd96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:31.691540 22639 raft_consensus.cc:2258] T a7851add5897418d9026e185cbe9db73 P 02f87f00febb4e59b2865dc593a5fd96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:31.691831 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 22185
W20260807 11:23:31.711601 22104 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.197.1:36407 (error 108)
I20260807 11:23:31.713002 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 22324
I20260807 11:23:31.713878 22645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:31.712961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:36407 (local address 127.0.0.1:45188)
0807 11:23:31.713522 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:31.713674 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:36407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":265,"threads_started":1}
W20260807 11:23:31.714382 22104 catalog_manager.cc:4584] TS 02f87f00febb4e59b2865dc593a5fd96 (127.16.197.1:36407): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a7851add5897418d9026e185cbe9db73: Network error: Client connection negotiation failed: client connection to 127.16.197.1:36407: connect: Connection refused (error 111)
I20260807 11:23:31.716285 22609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:31.713878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58518 (local address 127.16.197.2:38241)
0807 11:23:31.714065 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:31.714071 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:31.714095 (+ 24us) server_negotiation.cc:407] Connection header received
0807 11:23:31.714306 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:31.714311 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:31.714431 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:31.714623 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:31.715625 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:31.715629 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:31.715633 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:23:31.715648 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:31.715679 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:23:31.715691 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:31.715801 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:31.715916 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 11:23:31.715966 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 11:23:31.716957 22454 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:31.768296 22645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:31.767676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:36407 (local address 127.0.0.1:45192)
0807 11:23:31.767966 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:31.768094 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:36407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260807 11:23:31.769532 22645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:31.769092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:36407 (local address 127.0.0.1:45206)
0807 11:23:31.769264 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:31.769358 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:36407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 11:23:31.809648 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 22324
I20260807 11:23:31.829690 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 22462
I20260807 11:23:31.830849 22645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:31.830120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:36407 (local address 127.0.0.1:45212)
0807 11:23:31.830357 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:31.830466 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:36407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 11:23:31.832023 22645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:31.831572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:36407 (local address 127.0.0.1:45222)
0807 11:23:31.831761 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:31.831850 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:36407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 11:23:31.833570 22612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:31.830573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45538 (local address 127.16.197.3:39187)
0807 11:23:31.830741 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:31.830747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:31.830948 (+ 201us) server_negotiation.cc:407] Connection header received
0807 11:23:31.831538 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:31.831545 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:31.831627 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:31.831743 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:31.832833 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:31.832840 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:31.832845 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:23:31.832865 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:31.832908 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:23:31.832931 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:31.833091 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:31.833164 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 11:23:31.833237 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 11:23:31.834384 22591 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:31.912693 22645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:31.912111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:36407 (local address 127.0.0.1:45234)
0807 11:23:31.912384 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:31.912506 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:36407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260807 11:23:31.914230 22645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:31.913569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:36407 (local address 127.0.0.1:45242)
0807 11:23:31.913881 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:31.913999 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:36407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260807 11:23:31.959260 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 22462
I20260807 11:23:31.980412 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 22088
I20260807 11:23:31.981611 22152 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:32.098598 22645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:32.097009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:36407 (local address 127.0.0.1:45248)
0807 11:23:32.098216 (+ 1207us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:32.098372 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:36407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":893}
I20260807 11:23:32.098769 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 22088
2026-08-07T11:23:32Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2965 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 11:23:32.142895 17172 mini_oidc.cc:152] Starting JWKS server
I20260807 11:23:32.145876 17172 webserver.cc:460] Webserver started at https://127.0.0.1:38347/ using document root <none> and password file <none>
I20260807 11:23:32.146292 17172 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 11:23:32.147025 17172 webserver.cc:460] Webserver started at http://127.0.0.1:40929/ using document root <none> and password file <none>
2026-08-07T11:23:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:23:32Z Disabled control of system clock
I20260807 11:23:32.177768 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:33525
--webserver_interface=127.16.197.62
--webserver_port=0
--builtin_ntp_servers=127.16.197.20:36399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:33525
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38347/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786101765888975-17172-0/testchainca.cert with env {}
W20260807 11:23:32.510392 22654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:32.510704 22654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:32.510797 22654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:32.518224 22654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:23:32.518340 22654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:32.518378 22654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:32.518436 22654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:23:32.518487 22654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 11:23:32.518551 22654 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 11:23:32.518592 22654 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38347/jwks.json
I20260807 11:23:32.528972 22654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:36399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:33525
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:33525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38347/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786101765888975-17172-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:32.530299 22654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:32.531672 22654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:32.543924 22659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:32.543829 22662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:32.543893 22660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:32.545349 22654 server_base.cc:1034] running on GCE node
I20260807 11:23:32.546103 22654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:32.547640 22654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:32.548837 22654 hybrid_clock.cc:648] HybridClock initialized: now 1786101812548778 us; error 52 us; skew 500 ppm
I20260807 11:23:32.549186 22654 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 11:23:32.551522 22654 webserver.cc:460] Webserver started at http://127.16.197.62:35387/ using document root <none> and password file <none>
I20260807 11:23:32.552198 22654 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:32.552325 22654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:32.552670 22654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:32.555622 22654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "616edb9279e04039b7d1be1e93504868"
format_stamp: "Formatted at 2026-08-07 11:23:32 on dist-test-slave-2w3w"
server_key: "ac80788eb499319937f24efb0554c1af"
server_key_iv: "b9f07a5e58a6d22aa6730f069a27b11e"
server_key_version: "encryptionkey@0"
I20260807 11:23:32.556449 22654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "616edb9279e04039b7d1be1e93504868"
format_stamp: "Formatted at 2026-08-07 11:23:32 on dist-test-slave-2w3w"
server_key: "ac80788eb499319937f24efb0554c1af"
server_key_iv: "b9f07a5e58a6d22aa6730f069a27b11e"
server_key_version: "encryptionkey@0"
I20260807 11:23:32.561738 22654 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 11:23:32.565479 22669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:32.566933 22654 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 11:23:32.567184 22654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "616edb9279e04039b7d1be1e93504868"
format_stamp: "Formatted at 2026-08-07 11:23:32 on dist-test-slave-2w3w"
server_key: "ac80788eb499319937f24efb0554c1af"
server_key_iv: "b9f07a5e58a6d22aa6730f069a27b11e"
server_key_version: "encryptionkey@0"
I20260807 11:23:32.567427 22654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:32.580817 22654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:32.581696 22654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:32.582022 22654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:32.601512 22654 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:33525
I20260807 11:23:32.601521 22724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:33525 every 8 connection(s)
I20260807 11:23:32.603080 22654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:23:32.606900 22725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:32.609606 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 22654
I20260807 11:23:32.609861 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:23:32.610533 17172 external_mini_cluster.cc:1428] Setting key 86aa52a49eb31bb31dd864d12f7eeb85
I20260807 11:23:32.619606 22725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 616edb9279e04039b7d1be1e93504868: Bootstrap starting.
I20260807 11:23:32.623656 22725 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 616edb9279e04039b7d1be1e93504868: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:32.624819 22725 log.cc:824] T 00000000000000000000000000000000 P 616edb9279e04039b7d1be1e93504868: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:32.625259 22728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:32.612666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32984 (local address 127.16.197.62:33525)
0807 11:23:32.613691 (+ 1025us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:32.613703 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:32.613755 (+ 52us) server_negotiation.cc:407] Connection header received
0807 11:23:32.615560 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:32.615586 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:32.616094 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:32.616454 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:32.617166 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:32.618549 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:32.620416 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:32.621114 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:32.621272 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:32.622699 (+ 1427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:32.622706 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:32.622721 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:23:32.623345 (+ 624us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:32.623442 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:23:32.623466 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:32.623570 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:32.623873 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 11:23:32.624251 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":379,"threads_started":1}
I20260807 11:23:32.627987 22725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 616edb9279e04039b7d1be1e93504868: No bootstrap required, opened a new log
I20260807 11:23:32.629382 17172 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 11:23:32.637220 22725 raft_consensus.cc:348] T 00000000000000000000000000000000 P 616edb9279e04039b7d1be1e93504868 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "616edb9279e04039b7d1be1e93504868" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33525 } }
I20260807 11:23:32.637637 22725 raft_consensus.cc:374] T 00000000000000000000000000000000 P 616edb9279e04039b7d1be1e93504868 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:32.637717 22725 raft_consensus.cc:729] T 00000000000000000000000000000000 P 616edb9279e04039b7d1be1e93504868 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 616edb9279e04039b7d1be1e93504868, State: Initialized, Role: FOLLOWER
I20260807 11:23:32.638248 22725 consensus_queue.cc:260] T 00000000000000000000000000000000 P 616edb9279e04039b7d1be1e93504868 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "616edb9279e04039b7d1be1e93504868" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33525 } }
I20260807 11:23:32.638448 22725 raft_consensus.cc:388] T 00000000000000000000000000000000 P 616edb9279e04039b7d1be1e93504868 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:23:32.638564 22725 raft_consensus.cc:482] T 00000000000000000000000000000000 P 616edb9279e04039b7d1be1e93504868 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:23:32.638692 22725 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 616edb9279e04039b7d1be1e93504868 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:32.640898 22725 raft_consensus.cc:504] T 00000000000000000000000000000000 P 616edb9279e04039b7d1be1e93504868 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "616edb9279e04039b7d1be1e93504868" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33525 } }
I20260807 11:23:32.641366 22725 leader_election.cc:302] T 00000000000000000000000000000000 P 616edb9279e04039b7d1be1e93504868 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 616edb9279e04039b7d1be1e93504868; no voters:
I20260807 11:23:32.641906 22725 leader_election.cc:288] T 00000000000000000000000000000000 P 616edb9279e04039b7d1be1e93504868 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:23:32.642089 22730 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 616edb9279e04039b7d1be1e93504868 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:32.642825 22730 raft_consensus.cc:686] T 00000000000000000000000000000000 P 616edb9279e04039b7d1be1e93504868 [term 1 LEADER]: Becoming Leader. State: Replica: 616edb9279e04039b7d1be1e93504868, State: Running, Role: LEADER
I20260807 11:23:32.643381 22730 consensus_queue.cc:237] T 00000000000000000000000000000000 P 616edb9279e04039b7d1be1e93504868 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "616edb9279e04039b7d1be1e93504868" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33525 } }
I20260807 11:23:32.644266 22725 sys_catalog.cc:564] T 00000000000000000000000000000000 P 616edb9279e04039b7d1be1e93504868 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:23:32.646425 22731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 616edb9279e04039b7d1be1e93504868 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "616edb9279e04039b7d1be1e93504868" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "616edb9279e04039b7d1be1e93504868" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33525 } } }
I20260807 11:23:32.646874 22731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 616edb9279e04039b7d1be1e93504868 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:32.646400 22732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 616edb9279e04039b7d1be1e93504868 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 616edb9279e04039b7d1be1e93504868. Latest consensus state: current_term: 1 leader_uuid: "616edb9279e04039b7d1be1e93504868" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "616edb9279e04039b7d1be1e93504868" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33525 } } }
I20260807 11:23:32.647409 22732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 616edb9279e04039b7d1be1e93504868 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:32.648546 22736 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:23:32.653972 22736 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:23:32.664419 22736 catalog_manager.cc:1347] Generated new cluster ID: 8ca2fa53d3c64761bdff40f3bf8d2e78
I20260807 11:23:32.664546 22736 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:23:32.688233 22736 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:23:32.689603 22736 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:23:32.702708 22736 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 616edb9279e04039b7d1be1e93504868: Generated new TSK 0
I20260807 11:23:32.703670 22736 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 11:23:32.758085 22756 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 11:23:32.758322 22756 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 11:23:32.760322 22728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:32.729186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32998 (local address 127.16.197.62:33525)
0807 11:23:32.729398 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:32.729405 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:32.730028 (+ 623us) server_negotiation.cc:407] Connection header received
0807 11:23:32.730440 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:32.730444 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:32.730512 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:32.730645 (+ 133us) server_negotiation.cc:226] Negotiated authn=JWT
0807 11:23:32.731340 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:32.732059 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:32.734122 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:32.734595 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:32.734692 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:32.735376 (+ 684us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 11:23:32.759784 (+ 24408us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38347/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 11:23:32.760087 (+ 303us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:32998: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38347/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":69}
W20260807 11:23:32.760484 22728 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:32998: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38347/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 11:23:32.762673 22755 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:23:32.729076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33525 (local address 127.0.0.1:32998)
0807 11:23:32.729905 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:32.729925 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:32.730132 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:32.730701 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:32.730708 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:32.730723 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0807 11:23:32.731159 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:32.731173 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:32.732199 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:32.732206 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:32.733969 (+ 1763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:32.733984 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:32.734787 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:32.734795 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:32.735019 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:32.735105 (+ 86us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 11:23:32.760089 (+ 24984us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38347/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 11:23:32.760483 (+ 394us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.16.197.62:33525: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38347/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":463,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.16.197.62:33525: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38347/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 11:23:32.764989 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 22654
I20260807 11:23:32.766295 22720 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:32.863276 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 22654
2026-08-07T11:23:32Z chronyd exiting
I20260807 11:23:32.916272 17172 test_util.cc:175] -----------------------------------------------
I20260807 11:23:32.916436 17172 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786101765888975-17172-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (778 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 11:23:32.921628 17172 mini_oidc.cc:152] Starting JWKS server
I20260807 11:23:32.923609 17172 webserver.cc:460] Webserver started at https://127.0.0.1:41769/ using document root <none> and password file <none>
I20260807 11:23:32.923863 17172 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 11:23:32.924568 17172 webserver.cc:460] Webserver started at http://127.0.0.1:38731/ using document root <none> and password file <none>
2026-08-07T11:23:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:23:32Z Disabled control of system clock
I20260807 11:23:32.958110 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:42209
--webserver_interface=127.16.197.62
--webserver_port=0
--builtin_ntp_servers=127.16.197.20:34075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:42209
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41769/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786101765888975-17172-0/testchainca.cert with env {}
W20260807 11:23:33.301865 22763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:33.302255 22763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:33.302433 22763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:33.309943 22763 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:23:33.310101 22763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:33.310189 22763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:33.310276 22763 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:23:33.310357 22763 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 11:23:33.310448 22763 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 11:23:33.310523 22763 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41769/jwks.json
I20260807 11:23:33.320714 22763 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:34075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:42209
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:42209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41769/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786101765888975-17172-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:33.322163 22763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:33.323582 22763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:33.335983 22768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:33.335942 22771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:33.335896 22769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:33.336795 22763 server_base.cc:1034] running on GCE node
I20260807 11:23:33.337646 22763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:33.339337 22763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:33.340586 22763 hybrid_clock.cc:648] HybridClock initialized: now 1786101813340537 us; error 55 us; skew 500 ppm
I20260807 11:23:33.341041 22763 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 11:23:33.343489 22763 webserver.cc:460] Webserver started at http://127.16.197.62:37695/ using document root <none> and password file <none>
I20260807 11:23:33.344213 22763 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:33.344349 22763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:33.344686 22763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:33.347713 22763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "cdbc9c7fc0ad45908ea416a6554ea873"
format_stamp: "Formatted at 2026-08-07 11:23:33 on dist-test-slave-2w3w"
server_key: "91998af0233973db15f528becebe97fe"
server_key_iv: "906a5040ab2ca6b9355195505ceb582b"
server_key_version: "encryptionkey@0"
I20260807 11:23:33.348544 22763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "cdbc9c7fc0ad45908ea416a6554ea873"
format_stamp: "Formatted at 2026-08-07 11:23:33 on dist-test-slave-2w3w"
server_key: "91998af0233973db15f528becebe97fe"
server_key_iv: "906a5040ab2ca6b9355195505ceb582b"
server_key_version: "encryptionkey@0"
I20260807 11:23:33.353888 22763 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 11:23:33.357398 22779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:33.358865 22763 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 11:23:33.359118 22763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "cdbc9c7fc0ad45908ea416a6554ea873"
format_stamp: "Formatted at 2026-08-07 11:23:33 on dist-test-slave-2w3w"
server_key: "91998af0233973db15f528becebe97fe"
server_key_iv: "906a5040ab2ca6b9355195505ceb582b"
server_key_version: "encryptionkey@0"
I20260807 11:23:33.359354 22763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:33.376044 22763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:33.376875 22763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:33.377141 22763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:33.396601 22763 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:42209
I20260807 11:23:33.396615 22833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:42209 every 8 connection(s)
I20260807 11:23:33.398056 22763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:23:33.400887 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 22763
I20260807 11:23:33.401110 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:23:33.401976 17172 external_mini_cluster.cc:1428] Setting key bbb3a0da091359f13fdf0294e494bdd4
I20260807 11:23:33.402689 22834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:33.416199 22834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdbc9c7fc0ad45908ea416a6554ea873: Bootstrap starting.
I20260807 11:23:33.416957 22837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:33.404211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42282 (local address 127.16.197.62:42209)
0807 11:23:33.405267 (+ 1056us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:33.405286 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:33.405329 (+ 43us) server_negotiation.cc:407] Connection header received
0807 11:23:33.407705 (+ 2376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:33.407731 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:33.408249 (+ 518us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:33.408674 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:33.409313 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:33.410650 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:33.412043 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:33.412790 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:33.413006 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:33.414355 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:33.414363 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:33.414377 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:23:33.415039 (+ 662us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:33.415127 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:23:33.415150 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:33.415283 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:33.415520 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 11:23:33.415847 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":398,"threads_started":1}
I20260807 11:23:33.419622 22834 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cdbc9c7fc0ad45908ea416a6554ea873: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:33.420771 17172 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 11:23:33.420806 22834 log.cc:824] T 00000000000000000000000000000000 P cdbc9c7fc0ad45908ea416a6554ea873: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:33.423903 22834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdbc9c7fc0ad45908ea416a6554ea873: No bootstrap required, opened a new log
I20260807 11:23:33.432055 22834 raft_consensus.cc:348] T 00000000000000000000000000000000 P cdbc9c7fc0ad45908ea416a6554ea873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdbc9c7fc0ad45908ea416a6554ea873" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42209 } }
I20260807 11:23:33.432442 22834 raft_consensus.cc:374] T 00000000000000000000000000000000 P cdbc9c7fc0ad45908ea416a6554ea873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:33.432523 22834 raft_consensus.cc:729] T 00000000000000000000000000000000 P cdbc9c7fc0ad45908ea416a6554ea873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdbc9c7fc0ad45908ea416a6554ea873, State: Initialized, Role: FOLLOWER
I20260807 11:23:33.433022 22834 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdbc9c7fc0ad45908ea416a6554ea873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdbc9c7fc0ad45908ea416a6554ea873" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42209 } }
I20260807 11:23:33.433229 22834 raft_consensus.cc:388] T 00000000000000000000000000000000 P cdbc9c7fc0ad45908ea416a6554ea873 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:23:33.433341 22834 raft_consensus.cc:482] T 00000000000000000000000000000000 P cdbc9c7fc0ad45908ea416a6554ea873 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:23:33.433471 22834 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cdbc9c7fc0ad45908ea416a6554ea873 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:33.435746 22834 raft_consensus.cc:504] T 00000000000000000000000000000000 P cdbc9c7fc0ad45908ea416a6554ea873 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdbc9c7fc0ad45908ea416a6554ea873" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42209 } }
I20260807 11:23:33.436219 22834 leader_election.cc:302] T 00000000000000000000000000000000 P cdbc9c7fc0ad45908ea416a6554ea873 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cdbc9c7fc0ad45908ea416a6554ea873; no voters:
I20260807 11:23:33.436753 22834 leader_election.cc:288] T 00000000000000000000000000000000 P cdbc9c7fc0ad45908ea416a6554ea873 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:23:33.436941 22839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cdbc9c7fc0ad45908ea416a6554ea873 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:33.437649 22839 raft_consensus.cc:686] T 00000000000000000000000000000000 P cdbc9c7fc0ad45908ea416a6554ea873 [term 1 LEADER]: Becoming Leader. State: Replica: cdbc9c7fc0ad45908ea416a6554ea873, State: Running, Role: LEADER
I20260807 11:23:33.438191 22839 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdbc9c7fc0ad45908ea416a6554ea873 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdbc9c7fc0ad45908ea416a6554ea873" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42209 } }
I20260807 11:23:33.439007 22834 sys_catalog.cc:564] T 00000000000000000000000000000000 P cdbc9c7fc0ad45908ea416a6554ea873 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:23:33.441306 22841 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdbc9c7fc0ad45908ea416a6554ea873 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdbc9c7fc0ad45908ea416a6554ea873. Latest consensus state: current_term: 1 leader_uuid: "cdbc9c7fc0ad45908ea416a6554ea873" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdbc9c7fc0ad45908ea416a6554ea873" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42209 } } }
I20260807 11:23:33.441262 22840 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdbc9c7fc0ad45908ea416a6554ea873 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cdbc9c7fc0ad45908ea416a6554ea873" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdbc9c7fc0ad45908ea416a6554ea873" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42209 } } }
I20260807 11:23:33.441751 22841 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdbc9c7fc0ad45908ea416a6554ea873 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:33.441807 22840 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdbc9c7fc0ad45908ea416a6554ea873 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:33.442637 22846 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:23:33.448541 22846 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:23:33.458319 22846 catalog_manager.cc:1347] Generated new cluster ID: 66d67f47974d4de5bc55d2ed5d609bb4
I20260807 11:23:33.458454 22846 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:23:33.501689 22846 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:23:33.503175 22846 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:23:33.511950 22846 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cdbc9c7fc0ad45908ea416a6554ea873: Generated new TSK 0
I20260807 11:23:33.512831 22846 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 11:23:33.559306 22864 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 11:23:33.559448 22864 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260807 11:23:33.560871 22863 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:23:33.540233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42209 (local address 127.0.0.1:42286)
0807 11:23:33.540880 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:33.540896 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:33.541024 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:33.541559 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:33.541562 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:33.541571 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0807 11:23:33.541839 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:33.541846 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:33.543322 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:33.543326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:33.544384 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:33.544396 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:33.545397 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:33.545413 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:33.545544 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:33.545578 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 11:23:33.560340 (+ 14762us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41769/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 11:23:33.560574 (+ 234us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.16.197.62:42209: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41769/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":349,"threads_started":1}
I20260807 11:23:33.560882 22837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:33.540332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42286 (local address 127.16.197.62:42209)
0807 11:23:33.540539 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:33.540547 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:33.540987 (+ 440us) server_negotiation.cc:407] Connection header received
0807 11:23:33.541235 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:33.541241 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:33.541362 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:33.541529 (+ 167us) server_negotiation.cc:226] Negotiated authn=JWT
0807 11:23:33.541980 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:33.543184 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:33.544535 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:33.545237 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:33.545377 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:33.545888 (+ 511us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 11:23:33.560069 (+ 14181us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41769/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 11:23:33.560534 (+ 465us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42286: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41769/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":64}
W20260807 11:23:33.561009 22837 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42286: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41769/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260807 11:23:33.562634 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 22763
I20260807 11:23:33.564011 22829 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:33.662577 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 22763
2026-08-07T11:23:33Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (800 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 11:23:33.720963 17172 mini_oidc.cc:152] Starting JWKS server
I20260807 11:23:33.723007 17172 webserver.cc:460] Webserver started at https://127.0.0.1:32771/ using document root <none> and password file <none>
I20260807 11:23:33.723304 17172 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 11:23:33.723963 17172 webserver.cc:460] Webserver started at http://127.0.0.1:34387/ using document root <none> and password file <none>
2026-08-07T11:23:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:23:33Z Disabled control of system clock
I20260807 11:23:33.756129 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:35575
--webserver_interface=127.16.197.62
--webserver_port=0
--builtin_ntp_servers=127.16.197.20:41507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:35575
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32771/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 11:23:34.089936 22872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:34.090245 22872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:34.090407 22872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:34.097863 22872 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:23:34.098004 22872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:34.098094 22872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:34.098198 22872 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:23:34.098279 22872 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 11:23:34.098371 22872 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 11:23:34.098439 22872 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:32771/jwks.json
I20260807 11:23:34.111325 22872 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:41507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:35575
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:35575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32771/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:34.112715 22872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:34.114526 22872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:34.127462 22877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:34.127501 22880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:34.127417 22878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:34.128336 22872 server_base.cc:1034] running on GCE node
I20260807 11:23:34.129112 22872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:34.130473 22872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:34.131662 22872 hybrid_clock.cc:648] HybridClock initialized: now 1786101814131606 us; error 50 us; skew 500 ppm
I20260807 11:23:34.132023 22872 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 11:23:34.134310 22872 webserver.cc:460] Webserver started at http://127.16.197.62:42527/ using document root <none> and password file <none>
I20260807 11:23:34.135069 22872 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:34.135187 22872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:34.135522 22872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:34.138437 22872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "25e543dd679048ceae2227f36fb3d88b"
format_stamp: "Formatted at 2026-08-07 11:23:34 on dist-test-slave-2w3w"
server_key: "ec2db98dc54d28aa7a4ea38d1481a6cf"
server_key_iv: "5ff8ff3b3535d96474bca9bbd6977391"
server_key_version: "encryptionkey@0"
I20260807 11:23:34.139369 22872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "25e543dd679048ceae2227f36fb3d88b"
format_stamp: "Formatted at 2026-08-07 11:23:34 on dist-test-slave-2w3w"
server_key: "ec2db98dc54d28aa7a4ea38d1481a6cf"
server_key_iv: "5ff8ff3b3535d96474bca9bbd6977391"
server_key_version: "encryptionkey@0"
I20260807 11:23:34.144692 22872 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 11:23:34.148155 22886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:34.149554 22872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 11:23:34.149740 22872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "25e543dd679048ceae2227f36fb3d88b"
format_stamp: "Formatted at 2026-08-07 11:23:34 on dist-test-slave-2w3w"
server_key: "ec2db98dc54d28aa7a4ea38d1481a6cf"
server_key_iv: "5ff8ff3b3535d96474bca9bbd6977391"
server_key_version: "encryptionkey@0"
I20260807 11:23:34.149951 22872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:34.164031 22872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:34.164985 22872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:34.165302 22872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:34.184212 22872 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:35575
I20260807 11:23:34.184233 22942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:35575 every 8 connection(s)
I20260807 11:23:34.185590 22872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:23:34.187114 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 22872
I20260807 11:23:34.187354 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:23:34.188189 17172 external_mini_cluster.cc:1428] Setting key c60793a7ef670280506489a73eab8ce5
I20260807 11:23:34.190449 22943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:34.202066 22946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:34.190111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38120 (local address 127.16.197.62:35575)
0807 11:23:34.191153 (+ 1042us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:34.191171 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:34.191213 (+ 42us) server_negotiation.cc:407] Connection header received
0807 11:23:34.192788 (+ 1575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:34.192808 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:34.193261 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:34.193587 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:34.194176 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:34.195569 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:34.197535 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:34.198206 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:34.198349 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:34.199664 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:34.199671 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:34.199685 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:23:34.200277 (+ 592us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:34.200361 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:23:34.200388 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:34.200483 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:34.200731 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 11:23:34.201092 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":380,"threads_started":1}
I20260807 11:23:34.203583 22943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25e543dd679048ceae2227f36fb3d88b: Bootstrap starting.
I20260807 11:23:34.206063 17172 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 11:23:34.207065 22943 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25e543dd679048ceae2227f36fb3d88b: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:34.208057 22943 log.cc:824] T 00000000000000000000000000000000 P 25e543dd679048ceae2227f36fb3d88b: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:34.211302 22943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25e543dd679048ceae2227f36fb3d88b: No bootstrap required, opened a new log
I20260807 11:23:34.219162 22943 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25e543dd679048ceae2227f36fb3d88b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25e543dd679048ceae2227f36fb3d88b" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 35575 } }
I20260807 11:23:34.219578 22943 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25e543dd679048ceae2227f36fb3d88b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:34.219689 22943 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25e543dd679048ceae2227f36fb3d88b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e543dd679048ceae2227f36fb3d88b, State: Initialized, Role: FOLLOWER
I20260807 11:23:34.220254 22943 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25e543dd679048ceae2227f36fb3d88b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25e543dd679048ceae2227f36fb3d88b" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 35575 } }
I20260807 11:23:34.220502 22943 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25e543dd679048ceae2227f36fb3d88b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:23:34.220661 22943 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25e543dd679048ceae2227f36fb3d88b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:23:34.220845 22943 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25e543dd679048ceae2227f36fb3d88b [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:34.223088 22943 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25e543dd679048ceae2227f36fb3d88b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25e543dd679048ceae2227f36fb3d88b" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 35575 } }
I20260807 11:23:34.223603 22943 leader_election.cc:302] T 00000000000000000000000000000000 P 25e543dd679048ceae2227f36fb3d88b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25e543dd679048ceae2227f36fb3d88b; no voters:
I20260807 11:23:34.224234 22943 leader_election.cc:288] T 00000000000000000000000000000000 P 25e543dd679048ceae2227f36fb3d88b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:23:34.224416 22948 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25e543dd679048ceae2227f36fb3d88b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:34.225843 22948 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25e543dd679048ceae2227f36fb3d88b [term 1 LEADER]: Becoming Leader. State: Replica: 25e543dd679048ceae2227f36fb3d88b, State: Running, Role: LEADER
I20260807 11:23:34.226377 22948 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25e543dd679048ceae2227f36fb3d88b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25e543dd679048ceae2227f36fb3d88b" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 35575 } }
I20260807 11:23:34.226540 22943 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25e543dd679048ceae2227f36fb3d88b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:23:34.229430 22949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25e543dd679048ceae2227f36fb3d88b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25e543dd679048ceae2227f36fb3d88b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25e543dd679048ceae2227f36fb3d88b" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 35575 } } }
I20260807 11:23:34.229846 22949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25e543dd679048ceae2227f36fb3d88b [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:34.230340 22950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25e543dd679048ceae2227f36fb3d88b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25e543dd679048ceae2227f36fb3d88b. Latest consensus state: current_term: 1 leader_uuid: "25e543dd679048ceae2227f36fb3d88b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25e543dd679048ceae2227f36fb3d88b" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 35575 } } }
I20260807 11:23:34.230667 22950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25e543dd679048ceae2227f36fb3d88b [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:34.232283 22956 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:23:34.237912 22956 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:23:34.247123 22956 catalog_manager.cc:1347] Generated new cluster ID: cc269330e62b41f2a055964d37b05976
I20260807 11:23:34.247277 22956 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:23:34.275204 22956 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:23:34.277138 22956 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:23:34.296803 22956 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25e543dd679048ceae2227f36fb3d88b: Generated new TSK 0
I20260807 11:23:34.297958 22956 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:23:34.328845 22946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:34.320090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38132 (local address 127.16.197.62:35575)
0807 11:23:34.320319 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:34.320328 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:34.320777 (+ 449us) server_negotiation.cc:407] Connection header received
0807 11:23:34.321038 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:34.321042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:34.321118 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:34.321248 (+ 130us) server_negotiation.cc:226] Negotiated authn=JWT
0807 11:23:34.321819 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:34.322593 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:34.324645 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:34.325186 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:34.325280 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:34.325908 (+ 628us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 11:23:34.328166 (+ 2258us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32771/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 11:23:34.328523 (+ 357us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38132: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32771/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":46}
W20260807 11:23:34.328986 22946 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38132: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32771/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 11:23:34.329075 22973 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:23:34.319972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:35575 (local address 127.0.0.1:38132)
0807 11:23:34.320648 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:34.320674 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:34.320854 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:34.321294 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:34.321299 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:34.321314 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0807 11:23:34.321666 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:34.321678 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:34.322738 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:34.322743 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:34.324496 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:34.324512 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:34.325376 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:34.325384 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:34.325607 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:34.325653 (+ 46us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 11:23:34.328384 (+ 2731us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32771/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 11:23:34.328636 (+ 252us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.16.197.62:35575: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32771/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":368,"threads_started":1}
I20260807 11:23:34.329938 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 22872
I20260807 11:23:34.331175 22937 generic_service.cc:196] Checking for leaks (request via RPC)
E20260807 11:23:34.346825 22974 webserver.cc:533] Webserver: sslize failed:
E20260807 11:23:34.347004 22974 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 11:23:34.429672 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 22872
2026-08-07T11:23:34Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (766 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T11:23:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:23:34Z Disabled control of system clock
I20260807 11:23:34.518378 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:33681
--webserver_interface=127.16.197.62
--webserver_port=0
--builtin_ntp_servers=127.16.197.20:43079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:33681
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786101765888975-17172-0/testchainca.cert with env {}
W20260807 11:23:34.860460 22979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:34.860818 22979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:34.860975 22979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:34.868465 22979 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:23:34.868583 22979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:34.868669 22979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:34.868773 22979 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:23:34.868829 22979 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 11:23:34.868886 22979 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 11:23:34.868956 22979 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 11:23:34.879729 22979 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:43079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:33681
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:33681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786101765888975-17172-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:34.881168 22979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:34.882579 22979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:34.895552 22984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:34.895535 22987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:34.895558 22985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:34.896366 22979 server_base.cc:1034] running on GCE node
I20260807 11:23:34.897236 22979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:34.898695 22979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:34.899935 22979 hybrid_clock.cc:648] HybridClock initialized: now 1786101814899876 us; error 64 us; skew 500 ppm
I20260807 11:23:34.900295 22979 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 11:23:34.907244 22979 webserver.cc:460] Webserver started at http://127.16.197.62:40875/ using document root <none> and password file <none>
I20260807 11:23:34.908025 22979 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:34.908170 22979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:34.908499 22979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:34.912024 22979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "47d31b946a594b63a6fd7e975e21ac10"
format_stamp: "Formatted at 2026-08-07 11:23:34 on dist-test-slave-2w3w"
server_key: "cf5fdafc8c8df0475df1c1a73eee3579"
server_key_iv: "917086a38f012bdfbe9986d4a45b22dd"
server_key_version: "encryptionkey@0"
I20260807 11:23:34.912843 22979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "47d31b946a594b63a6fd7e975e21ac10"
format_stamp: "Formatted at 2026-08-07 11:23:34 on dist-test-slave-2w3w"
server_key: "cf5fdafc8c8df0475df1c1a73eee3579"
server_key_iv: "917086a38f012bdfbe9986d4a45b22dd"
server_key_version: "encryptionkey@0"
I20260807 11:23:34.918119 22979 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 11:23:34.921751 22993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:34.923143 22979 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:23:34.923338 22979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "47d31b946a594b63a6fd7e975e21ac10"
format_stamp: "Formatted at 2026-08-07 11:23:34 on dist-test-slave-2w3w"
server_key: "cf5fdafc8c8df0475df1c1a73eee3579"
server_key_iv: "917086a38f012bdfbe9986d4a45b22dd"
server_key_version: "encryptionkey@0"
I20260807 11:23:34.923569 22979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:34.952001 22979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:34.952951 22979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:34.953277 22979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:34.973811 22979 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:33681
I20260807 11:23:34.973821 23048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:33681 every 8 connection(s)
I20260807 11:23:34.975490 22979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:23:34.979898 23051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:34.981172 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 22979
I20260807 11:23:34.981370 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:23:34.982134 17172 external_mini_cluster.cc:1428] Setting key e575f0d6a6a7da6d77dbeb8d14c41f53
I20260807 11:23:34.996158 23051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47d31b946a594b63a6fd7e975e21ac10: Bootstrap starting.
I20260807 11:23:34.999262 23054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:34.984321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34528 (local address 127.16.197.62:33681)
0807 11:23:34.985525 (+ 1204us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:34.985541 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:34.985597 (+ 56us) server_negotiation.cc:407] Connection header received
0807 11:23:34.987446 (+ 1849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:34.987482 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:34.988191 (+ 709us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:34.988700 (+ 509us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:34.989256 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:34.991307 (+ 2051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:34.992704 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:34.993932 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:34.994129 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:34.995729 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:34.995741 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:34.995834 (+ 93us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:23:34.996741 (+ 907us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:34.996907 (+ 166us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:23:34.996951 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:34.997090 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:34.997323 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 11:23:34.997698 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":363,"threads_started":1}
I20260807 11:23:35.001037 23051 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47d31b946a594b63a6fd7e975e21ac10: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:35.002614 23051 log.cc:824] T 00000000000000000000000000000000 P 47d31b946a594b63a6fd7e975e21ac10: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:35.004226 17172 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 11:23:35.006110 23051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47d31b946a594b63a6fd7e975e21ac10: No bootstrap required, opened a new log
I20260807 11:23:35.015156 23051 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47d31b946a594b63a6fd7e975e21ac10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47d31b946a594b63a6fd7e975e21ac10" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33681 } }
I20260807 11:23:35.015645 23051 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47d31b946a594b63a6fd7e975e21ac10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:35.015759 23051 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47d31b946a594b63a6fd7e975e21ac10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47d31b946a594b63a6fd7e975e21ac10, State: Initialized, Role: FOLLOWER
I20260807 11:23:35.016390 23051 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47d31b946a594b63a6fd7e975e21ac10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47d31b946a594b63a6fd7e975e21ac10" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33681 } }
I20260807 11:23:35.016669 23051 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47d31b946a594b63a6fd7e975e21ac10 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:23:35.016821 23051 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47d31b946a594b63a6fd7e975e21ac10 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:23:35.017010 23051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47d31b946a594b63a6fd7e975e21ac10 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:35.019536 23051 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47d31b946a594b63a6fd7e975e21ac10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47d31b946a594b63a6fd7e975e21ac10" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33681 } }
I20260807 11:23:35.020220 23051 leader_election.cc:302] T 00000000000000000000000000000000 P 47d31b946a594b63a6fd7e975e21ac10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47d31b946a594b63a6fd7e975e21ac10; no voters:
I20260807 11:23:35.020998 23051 leader_election.cc:288] T 00000000000000000000000000000000 P 47d31b946a594b63a6fd7e975e21ac10 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:23:35.021229 23056 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47d31b946a594b63a6fd7e975e21ac10 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:35.022891 23056 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47d31b946a594b63a6fd7e975e21ac10 [term 1 LEADER]: Becoming Leader. State: Replica: 47d31b946a594b63a6fd7e975e21ac10, State: Running, Role: LEADER
I20260807 11:23:35.023638 23056 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47d31b946a594b63a6fd7e975e21ac10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47d31b946a594b63a6fd7e975e21ac10" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33681 } }
I20260807 11:23:35.023940 23051 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47d31b946a594b63a6fd7e975e21ac10 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:23:35.026975 23058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47d31b946a594b63a6fd7e975e21ac10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47d31b946a594b63a6fd7e975e21ac10. Latest consensus state: current_term: 1 leader_uuid: "47d31b946a594b63a6fd7e975e21ac10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47d31b946a594b63a6fd7e975e21ac10" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33681 } } }
I20260807 11:23:35.027433 23058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47d31b946a594b63a6fd7e975e21ac10 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:35.028051 23057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47d31b946a594b63a6fd7e975e21ac10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47d31b946a594b63a6fd7e975e21ac10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47d31b946a594b63a6fd7e975e21ac10" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33681 } } }
I20260807 11:23:35.028373 23057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47d31b946a594b63a6fd7e975e21ac10 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:35.031061 23064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:23:35.037263 23064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:23:35.053289 23064 catalog_manager.cc:1347] Generated new cluster ID: a93212f7ed684b4cb5476787201ae712
I20260807 11:23:35.053772 23064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:23:35.087796 23064 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:23:35.089934 23064 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:23:35.099557 23064 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47d31b946a594b63a6fd7e975e21ac10: Generated new TSK 0
I20260807 11:23:35.100946 23064 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:23:35.169257 23054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:35.122496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34536 (local address 127.16.197.62:33681)
0807 11:23:35.122723 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:35.122730 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:35.123161 (+ 431us) server_negotiation.cc:407] Connection header received
0807 11:23:35.123412 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:35.123418 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:35.123529 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:35.123727 (+ 198us) server_negotiation.cc:226] Negotiated authn=JWT
0807 11:23:35.124276 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:35.125522 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:35.127224 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:35.128033 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:35.128191 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:35.128813 (+ 622us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 11:23:35.168429 (+ 39616us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 11:23:35.168920 (+ 491us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34536: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":88}
W20260807 11:23:35.169435 23054 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34536: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 11:23:35.169502 23081 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:23:35.122328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33681 (local address 127.0.0.1:34536)
0807 11:23:35.123040 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:35.123064 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:35.123213 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:35.123833 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:35.123837 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:35.123848 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0807 11:23:35.124108 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:35.124117 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:35.125700 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:35.125705 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:35.126943 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:35.127029 (+ 86us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:35.128227 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:35.128233 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:35.128407 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:35.128467 (+ 60us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 11:23:35.168836 (+ 40369us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 11:23:35.169100 (+ 264us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.16.197.62:33681: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":341,"threads_started":1}
I20260807 11:23:35.170548 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 22979
I20260807 11:23:35.171958 23043 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:35.271607 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 22979
2026-08-07T11:23:35Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (821 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (432 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (446 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:23:36 dist-test-slave-2w3w krb5kdc[23097](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:23:36 dist-test-slave-2w3w krb5kdc[23097](info): set up 2 sockets
Aug 07 11:23:36 dist-test-slave-2w3w krb5kdc[23097](info): commencing operation
krb5kdc: starting...
W20260807 11:23:37.277154 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:23:37 dist-test-slave-2w3w krb5kdc[23097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:23:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:23:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:23:37.511445 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:38239
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:40351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:38239
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:37.849702 23111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:37.850046 23111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:37.850222 23111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:37.857926 23111 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:23:37.858064 23111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:37.858158 23111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:37.858251 23111 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:23:37.858352 23111 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:23:37.869089 23111 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:40351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:38239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:38239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:37.870661 23111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:37.872247 23111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:37.884968 23116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:37.885042 23119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:37.885046 23117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:37.886250 23111 server_base.cc:1034] running on GCE node
I20260807 11:23:37.887161 23111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:37.888756 23111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:37.889971 23111 hybrid_clock.cc:648] HybridClock initialized: now 1786101817889912 us; error 66 us; skew 500 ppm
I20260807 11:23:37.890360 23111 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:37 dist-test-slave-2w3w krb5kdc[23097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101817, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:37.893718 23111 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:23:37.895749 23111 webserver.cc:460] Webserver started at http://127.16.197.62:34113/ using document root <none> and password file <none>
I20260807 11:23:37.896517 23111 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:37.896656 23111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:37.897008 23111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:37.900279 23111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "5a4d8ddf704240df96ea21e4ed33c986"
format_stamp: "Formatted at 2026-08-07 11:23:37 on dist-test-slave-2w3w"
server_key: "f73eb452cbb36a989abadc1632e0809a"
server_key_iv: "6a890046caf83f928f74069914fa220d"
server_key_version: "encryptionkey@0"
I20260807 11:23:37.901139 23111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "5a4d8ddf704240df96ea21e4ed33c986"
format_stamp: "Formatted at 2026-08-07 11:23:37 on dist-test-slave-2w3w"
server_key: "f73eb452cbb36a989abadc1632e0809a"
server_key_iv: "6a890046caf83f928f74069914fa220d"
server_key_version: "encryptionkey@0"
I20260807 11:23:37.906756 23111 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260807 11:23:37.910568 23126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:37.912142 23111 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:23:37.912352 23111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "5a4d8ddf704240df96ea21e4ed33c986"
format_stamp: "Formatted at 2026-08-07 11:23:37 on dist-test-slave-2w3w"
server_key: "f73eb452cbb36a989abadc1632e0809a"
server_key_iv: "6a890046caf83f928f74069914fa220d"
server_key_version: "encryptionkey@0"
I20260807 11:23:37.912577 23111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:37.940867 23111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:37.945688 23111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:37.946031 23111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:37.967307 23111 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:38239
I20260807 11:23:37.967360 23179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:38239 every 8 connection(s)
I20260807 11:23:37.968849 23111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:23:37.973155 23180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:37.974913 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 23111
I20260807 11:23:37.975176 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:23:37.976060 17172 external_mini_cluster.cc:1428] Setting key dd149e78e19940b2b090f63c18caaab0
I20260807 11:23:37.989313 23180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a4d8ddf704240df96ea21e4ed33c986: Bootstrap starting.
Aug 07 11:23:37 dist-test-slave-2w3w krb5kdc[23097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:37.993409 23180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a4d8ddf704240df96ea21e4ed33c986: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:37.994534 23180 log.cc:824] T 00000000000000000000000000000000 P 5a4d8ddf704240df96ea21e4ed33c986: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:37.997658 23180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a4d8ddf704240df96ea21e4ed33c986: No bootstrap required, opened a new log
I20260807 11:23:38.001547 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:37.978353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41226 (local address 127.16.197.62:38239)
0807 11:23:37.979490 (+ 1137us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:37.979502 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:37.979550 (+ 48us) server_negotiation.cc:407] Connection header received
0807 11:23:37.981197 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:37.981223 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:37.981674 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:37.982039 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:37.984012 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:37.985611 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:37.987517 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:37.988368 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:37.988533 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:37.993513 (+ 4980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:37.993522 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:37.993558 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:37.993618 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:37.996976 (+ 3358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:37.997766 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:37.997796 (+ 30us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:37.997805 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:37.997994 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:37.998432 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:37.998436 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:37.998438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:37.999128 (+ 690us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:37.999507 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:37.999981 (+ 474us) server_negotiation.cc:299] Negotiation successful
0807 11:23:38.000461 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":901,"thread_start_us":402,"threads_started":1}
I20260807 11:23:38.006057 23180 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a4d8ddf704240df96ea21e4ed33c986 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4d8ddf704240df96ea21e4ed33c986" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 38239 } }
I20260807 11:23:38.006458 23180 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a4d8ddf704240df96ea21e4ed33c986 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:38.006572 23180 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a4d8ddf704240df96ea21e4ed33c986 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a4d8ddf704240df96ea21e4ed33c986, State: Initialized, Role: FOLLOWER
I20260807 11:23:38.007225 23180 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a4d8ddf704240df96ea21e4ed33c986 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4d8ddf704240df96ea21e4ed33c986" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 38239 } }
I20260807 11:23:38.007481 23180 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a4d8ddf704240df96ea21e4ed33c986 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:23:38.007630 23180 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a4d8ddf704240df96ea21e4ed33c986 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:23:38.007812 23180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a4d8ddf704240df96ea21e4ed33c986 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:38.010205 23180 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a4d8ddf704240df96ea21e4ed33c986 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4d8ddf704240df96ea21e4ed33c986" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 38239 } }
I20260807 11:23:38.010826 23180 leader_election.cc:302] T 00000000000000000000000000000000 P 5a4d8ddf704240df96ea21e4ed33c986 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a4d8ddf704240df96ea21e4ed33c986; no voters:
I20260807 11:23:38.011572 23180 leader_election.cc:288] T 00000000000000000000000000000000 P 5a4d8ddf704240df96ea21e4ed33c986 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:23:38.011785 23185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a4d8ddf704240df96ea21e4ed33c986 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:38.012620 23185 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a4d8ddf704240df96ea21e4ed33c986 [term 1 LEADER]: Becoming Leader. State: Replica: 5a4d8ddf704240df96ea21e4ed33c986, State: Running, Role: LEADER
I20260807 11:23:38.013250 23185 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a4d8ddf704240df96ea21e4ed33c986 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4d8ddf704240df96ea21e4ed33c986" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 38239 } }
I20260807 11:23:38.013870 23180 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a4d8ddf704240df96ea21e4ed33c986 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:23:38.016275 23187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a4d8ddf704240df96ea21e4ed33c986 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a4d8ddf704240df96ea21e4ed33c986. Latest consensus state: current_term: 1 leader_uuid: "5a4d8ddf704240df96ea21e4ed33c986" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4d8ddf704240df96ea21e4ed33c986" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 38239 } } }
I20260807 11:23:38.016714 23187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a4d8ddf704240df96ea21e4ed33c986 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:38.017390 23186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a4d8ddf704240df96ea21e4ed33c986 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a4d8ddf704240df96ea21e4ed33c986" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4d8ddf704240df96ea21e4ed33c986" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 38239 } } }
I20260807 11:23:38.017740 23186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a4d8ddf704240df96ea21e4ed33c986 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:38.020107 23194 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:23:38.026149 23194 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:23:38.036782 23194 catalog_manager.cc:1347] Generated new cluster ID: 754ade869986419085654df3769b54da
I20260807 11:23:38.036988 23194 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:23:38.051249 23194 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:23:38.052686 23194 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:23:38.060101 23194 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a4d8ddf704240df96ea21e4ed33c986: Generated new TSK 0
I20260807 11:23:38.061123 23194 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.197.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
I20260807 11:23:38.122277 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:0
--local_ip_for_outbound_sockets=127.16.197.1
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:38239
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:40351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:38.464016 23207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:38.464314 23207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:38.464493 23207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:38.471872 23207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:38.472019 23207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:38.472213 23207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:23:38.482396 23207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:40351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:38239
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:38.483841 23207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:38.485332 23207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:38.499327 23212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:38.499275 23213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:38.499711 23216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:38.500218 23207 server_base.cc:1034] running on GCE node
I20260807 11:23:38.501058 23207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:38.502035 23207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:38.503295 23207 hybrid_clock.cc:648] HybridClock initialized: now 1786101818503222 us; error 59 us; skew 500 ppm
I20260807 11:23:38.503650 23207 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:38 dist-test-slave-2w3w krb5kdc[23097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101818, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:38.506878 23207 init.cc:377] Logged in from keytab as kudu/127.16.197.1@KRBTEST.COM (short username kudu)
I20260807 11:23:38.508392 23207 webserver.cc:460] Webserver started at http://127.16.197.1:43261/ using document root <none> and password file <none>
I20260807 11:23:38.509063 23207 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:38.509166 23207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:38.509428 23207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:38.512454 23207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-0/data/instance:
uuid: "ceaa43b771154abc9246a91bd3e192ce"
format_stamp: "Formatted at 2026-08-07 11:23:38 on dist-test-slave-2w3w"
server_key: "f7168df4c1c02c2da1f494e68735e6b2"
server_key_iv: "5bbdf8bb65efc3e99d141e0b3f65595b"
server_key_version: "encryptionkey@0"
I20260807 11:23:38.513195 23207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance:
uuid: "ceaa43b771154abc9246a91bd3e192ce"
format_stamp: "Formatted at 2026-08-07 11:23:38 on dist-test-slave-2w3w"
server_key: "f7168df4c1c02c2da1f494e68735e6b2"
server_key_iv: "5bbdf8bb65efc3e99d141e0b3f65595b"
server_key_version: "encryptionkey@0"
I20260807 11:23:38.518829 23207 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 11:23:38.522470 23224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:38.523952 23207 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:23:38.524142 23207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "ceaa43b771154abc9246a91bd3e192ce"
format_stamp: "Formatted at 2026-08-07 11:23:38 on dist-test-slave-2w3w"
server_key: "f7168df4c1c02c2da1f494e68735e6b2"
server_key_iv: "5bbdf8bb65efc3e99d141e0b3f65595b"
server_key_version: "encryptionkey@0"
I20260807 11:23:38.524354 23207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:38.544399 23207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:38.548514 23207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:38.548820 23207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:38.549817 23207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:23:38.551322 23207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:38.551437 23207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:38.551605 23207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:38.551673 23207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:38.585826 23207 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:34213
I20260807 11:23:38.585893 23343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:34213 every 8 connection(s)
I20260807 11:23:38.587347 23207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:23:38.596736 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 23207
I20260807 11:23:38.597021 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance
I20260807 11:23:38.597847 17172 external_mini_cluster.cc:1428] Setting key dd3ca7deebea06078bdebeccad1fcc98
Aug 07 11:23:38 dist-test-slave-2w3w krb5kdc[23097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101818, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:38.612005 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:38.590211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:38359 (local address 127.16.197.62:38239)
0807 11:23:38.590869 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:38.590875 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:38.591976 (+ 1101us) server_negotiation.cc:407] Connection header received
0807 11:23:38.593898 (+ 1922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:38.593904 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:38.593980 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:38.594086 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:38.596073 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:38.597302 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:38.599587 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:38.600158 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:38.600316 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:38.604308 (+ 3992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:38.604317 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:38.604325 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:38.604385 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:38.608176 (+ 3791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:38.609364 (+ 1188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:38.609368 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:38.609371 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:38.609495 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:38.610084 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:38.610087 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:38.610089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:38.610368 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:38.610486 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:38.611555 (+ 1069us) server_negotiation.cc:299] Negotiation successful
0807 11:23:38.611748 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":358,"threads_started":1}
I20260807 11:23:38.613850 23347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:38.591018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:38239 (local address 127.16.197.1:38359)
0807 11:23:38.591804 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:38.591845 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:38.593568 (+ 1723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:38.594389 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:38.594399 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:38.595162 (+ 763us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:38.595840 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:38.595863 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:38.597502 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:38.597507 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:38.599397 (+ 1890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:38.599415 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:38.600442 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:38.600449 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:38.600683 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:38.601485 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:38.601518 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:38.604023 (+ 2505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:38.608369 (+ 4346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:38.608383 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:38.608408 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:38.609213 (+ 805us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:38.609615 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:38.609624 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:38.609631 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:38.609962 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:38.610505 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:38.610517 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:38.611107 (+ 590us) client_negotiation.cc:769] Sending connection context
0807 11:23:38.611454 (+ 347us) client_negotiation.cc:240] Negotiation successful
0807 11:23:38.612130 (+ 676us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":297,"threads_started":1}
I20260807 11:23:38.615643 23344 heartbeater.cc:347] Connected to a master server at 127.16.197.62:38239
I20260807 11:23:38.616016 23344 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:38.617075 23344 heartbeater.cc:511] Master 127.16.197.62:38239 requested a full tablet report, sending...
I20260807 11:23:38.619454 23144 ts_manager.cc:194] Registered new tserver with Master: ceaa43b771154abc9246a91bd3e192ce (127.16.197.1:34213)
I20260807 11:23:38.621114 23144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.1@KRBTEST.COM'} at 127.16.197.1:38359
WARNING: no policy specified for kudu/127.16.197.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
I20260807 11:23:38.655148 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:0
--local_ip_for_outbound_sockets=127.16.197.2
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:38239
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:40351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:38.998824 23350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:38.999195 23350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:38.999384 23350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:39.006840 23350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:39.007004 23350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:39.007200 23350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:23:39.017665 23350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:40351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:38239
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:39.019093 23350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:39.020642 23350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:39.034869 23359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:39.034991 23355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:39.034947 23357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:39.035869 23350 server_base.cc:1034] running on GCE node
I20260807 11:23:39.036613 23350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:39.037636 23350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:39.038870 23350 hybrid_clock.cc:648] HybridClock initialized: now 1786101819038822 us; error 40 us; skew 500 ppm
I20260807 11:23:39.039224 23350 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:39 dist-test-slave-2w3w krb5kdc[23097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101819, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:39.042421 23350 init.cc:377] Logged in from keytab as kudu/127.16.197.2@KRBTEST.COM (short username kudu)
I20260807 11:23:39.044001 23350 webserver.cc:460] Webserver started at http://127.16.197.2:44635/ using document root <none> and password file <none>
I20260807 11:23:39.044860 23350 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:39.045020 23350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:39.045343 23350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:39.048626 23350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-1/data/instance:
uuid: "bd04dde67f404f65a3b47580ed8faf39"
format_stamp: "Formatted at 2026-08-07 11:23:39 on dist-test-slave-2w3w"
server_key: "23e65f4c89582ab7fd93165d1be66e71"
server_key_iv: "cebe18cbe8c23a257b96a3eb0ff2a553"
server_key_version: "encryptionkey@0"
I20260807 11:23:39.049607 23350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance:
uuid: "bd04dde67f404f65a3b47580ed8faf39"
format_stamp: "Formatted at 2026-08-07 11:23:39 on dist-test-slave-2w3w"
server_key: "23e65f4c89582ab7fd93165d1be66e71"
server_key_iv: "cebe18cbe8c23a257b96a3eb0ff2a553"
server_key_version: "encryptionkey@0"
I20260807 11:23:39.056741 23350 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.003s
I20260807 11:23:39.060642 23366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:39.062057 23350 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 11:23:39.062251 23350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "bd04dde67f404f65a3b47580ed8faf39"
format_stamp: "Formatted at 2026-08-07 11:23:39 on dist-test-slave-2w3w"
server_key: "23e65f4c89582ab7fd93165d1be66e71"
server_key_iv: "cebe18cbe8c23a257b96a3eb0ff2a553"
server_key_version: "encryptionkey@0"
I20260807 11:23:39.062495 23350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:39.073652 23350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:39.078197 23350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:39.078500 23350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:39.079536 23350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:23:39.080838 23350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:39.080940 23350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:39.081080 23350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:39.081149 23350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:39.120025 23350 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:37145
I20260807 11:23:39.120092 23483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:37145 every 8 connection(s)
I20260807 11:23:39.121723 23350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:23:39.128371 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 23350
I20260807 11:23:39.128544 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance
I20260807 11:23:39.129012 17172 external_mini_cluster.cc:1428] Setting key 09cc7566a372009dd7b93c7731cc445b
Aug 07 11:23:39 dist-test-slave-2w3w krb5kdc[23097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101819, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:39.143855 23488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:39.124351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:53463 (local address 127.16.197.62:38239)
0807 11:23:39.124774 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:39.124779 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:39.125809 (+ 1030us) server_negotiation.cc:407] Connection header received
0807 11:23:39.127788 (+ 1979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:39.127794 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:39.127887 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:39.128040 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:39.129890 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:39.130947 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:39.132561 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:39.132994 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:39.133179 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:39.136809 (+ 3630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:39.136814 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:39.136817 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:39.136858 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:39.140290 (+ 3432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:39.141110 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:39.141118 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:39.141124 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:39.141310 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:39.141740 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:39.141748 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:39.141754 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:39.142176 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:39.142341 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:39.143067 (+ 726us) server_negotiation.cc:299] Negotiation successful
0807 11:23:39.143335 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":202,"threads_started":1}
I20260807 11:23:39.144556 23489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:39.124946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:38239 (local address 127.16.197.2:53463)
0807 11:23:39.125646 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:39.125686 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:39.127475 (+ 1789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:39.128253 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:39.128262 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:39.128997 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:39.129677 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:39.129697 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:39.131107 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:39.131111 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:39.132395 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:39.132413 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:39.133273 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:39.133282 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:39.133541 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:39.134251 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:39.134276 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:39.136558 (+ 2282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:39.140463 (+ 3905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:39.140471 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:39.140490 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:39.140969 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:39.141431 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:39.141434 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:39.141436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:39.141620 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:39.142306 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:39.142312 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:39.142643 (+ 331us) client_negotiation.cc:769] Sending connection context
0807 11:23:39.142888 (+ 245us) client_negotiation.cc:240] Negotiation successful
0807 11:23:39.143351 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":276,"threads_started":1}
I20260807 11:23:39.146056 23485 heartbeater.cc:347] Connected to a master server at 127.16.197.62:38239
I20260807 11:23:39.146457 23485 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:39.147375 23485 heartbeater.cc:511] Master 127.16.197.62:38239 requested a full tablet report, sending...
I20260807 11:23:39.149080 23144 ts_manager.cc:194] Registered new tserver with Master: bd04dde67f404f65a3b47580ed8faf39 (127.16.197.2:37145)
I20260807 11:23:39.149919 23144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.2@KRBTEST.COM'} at 127.16.197.2:53463
WARNING: no policy specified for kudu/127.16.197.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
I20260807 11:23:39.182529 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:0
--local_ip_for_outbound_sockets=127.16.197.3
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:38239
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:40351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:39.526602 23492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:39.526926 23492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:39.527089 23492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:39.534654 23492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:39.534844 23492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:39.535094 23492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:23:39.545194 23492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:40351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:38239
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:39.546607 23492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:39.548071 23492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:39.561439 23499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:39.561522 23498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:39.561913 23501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:39.563319 23492 server_base.cc:1034] running on GCE node
I20260807 11:23:39.563932 23492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:39.564915 23492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:39.566111 23492 hybrid_clock.cc:648] HybridClock initialized: now 1786101819566048 us; error 54 us; skew 500 ppm
I20260807 11:23:39.566476 23492 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:39 dist-test-slave-2w3w krb5kdc[23097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101819, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:39.569846 23492 init.cc:377] Logged in from keytab as kudu/127.16.197.3@KRBTEST.COM (short username kudu)
I20260807 11:23:39.571470 23492 webserver.cc:460] Webserver started at http://127.16.197.3:41965/ using document root <none> and password file <none>
I20260807 11:23:39.572155 23492 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:39.572256 23492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:39.572531 23492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:39.575654 23492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-2/data/instance:
uuid: "2b052d635ada4adb8c3efcdcc7996956"
format_stamp: "Formatted at 2026-08-07 11:23:39 on dist-test-slave-2w3w"
server_key: "1ef8575ac9b6a0ddcde6465acadf95a4"
server_key_iv: "e3dd04cc1267cf59283b026297d4c91a"
server_key_version: "encryptionkey@0"
I20260807 11:23:39.576484 23492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance:
uuid: "2b052d635ada4adb8c3efcdcc7996956"
format_stamp: "Formatted at 2026-08-07 11:23:39 on dist-test-slave-2w3w"
server_key: "1ef8575ac9b6a0ddcde6465acadf95a4"
server_key_iv: "e3dd04cc1267cf59283b026297d4c91a"
server_key_version: "encryptionkey@0"
I20260807 11:23:39.581838 23492 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 11:23:39.585513 23509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:39.587184 23492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 11:23:39.587369 23492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "2b052d635ada4adb8c3efcdcc7996956"
format_stamp: "Formatted at 2026-08-07 11:23:39 on dist-test-slave-2w3w"
server_key: "1ef8575ac9b6a0ddcde6465acadf95a4"
server_key_iv: "e3dd04cc1267cf59283b026297d4c91a"
server_key_version: "encryptionkey@0"
I20260807 11:23:39.587633 23492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:39.611577 23492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:39.616721 23492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:39.617014 23492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:39.618163 23492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:23:39.619705 23492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:39.619784 23492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:39.619921 23492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:39.619971 23492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:39.625218 23344 heartbeater.cc:503] Master 127.16.197.62:38239 was elected leader, sending a full tablet report...
I20260807 11:23:39.655258 23492 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:42585
I20260807 11:23:39.655316 23627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:42585 every 8 connection(s)
I20260807 11:23:39.656580 23492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:23:39.665925 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 23492
I20260807 11:23:39.666108 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance
I20260807 11:23:39.666591 17172 external_mini_cluster.cc:1428] Setting key 34d27d70e39c8af7e7cc6c70e0f5bf8e
Aug 07 11:23:39 dist-test-slave-2w3w krb5kdc[23097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101819, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:23:39.676394 23630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:39.659171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:37371 (local address 127.16.197.62:38239)
0807 11:23:39.659645 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:39.659650 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:39.660495 (+ 845us) server_negotiation.cc:407] Connection header received
0807 11:23:39.662299 (+ 1804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:39.662304 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:39.662410 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:39.662533 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:39.664151 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:39.664936 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:39.666704 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:39.667200 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:39.667392 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:39.671118 (+ 3726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:39.671123 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:39.671126 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:39.671159 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:39.673575 (+ 2416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:39.674277 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:39.674281 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:39.674283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:39.674402 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:39.674787 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:39.674790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:39.674792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:39.675129 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:39.675253 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:39.675948 (+ 695us) server_negotiation.cc:299] Negotiation successful
0807 11:23:39.676124 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":250,"threads_started":1}
I20260807 11:23:39.677443 23631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:39.659621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:38239 (local address 127.16.197.3:37371)
0807 11:23:39.660339 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:39.660377 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:39.662030 (+ 1653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:39.662766 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:39.662774 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:39.663375 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:39.663968 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:39.663987 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:39.665095 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:39.665100 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:39.666551 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:39.666568 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:39.667474 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:39.667481 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:39.667693 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:39.668484 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:39.668532 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:39.670864 (+ 2332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:39.673710 (+ 2846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:39.673717 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:39.673731 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:39.674166 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:39.674490 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:39.674493 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:39.674496 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:39.674679 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:39.675236 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:39.675241 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:39.675579 (+ 338us) client_negotiation.cc:769] Sending connection context
0807 11:23:39.675818 (+ 239us) client_negotiation.cc:240] Negotiation successful
0807 11:23:39.676219 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":291,"threads_started":1}
I20260807 11:23:39.678911 23628 heartbeater.cc:347] Connected to a master server at 127.16.197.62:38239
I20260807 11:23:39.679287 23628 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:39.680024 23628 heartbeater.cc:511] Master 127.16.197.62:38239 requested a full tablet report, sending...
I20260807 11:23:39.681641 23144 ts_manager.cc:194] Registered new tserver with Master: 2b052d635ada4adb8c3efcdcc7996956 (127.16.197.3:42585)
I20260807 11:23:39.682677 17172 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 11:23:39.682698 23144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.3@KRBTEST.COM'} at 127.16.197.3:37371
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/test-admin.keytab.
Aug 07 11:23:39 dist-test-slave-2w3w krb5kdc[23097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:39.713246 17172 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 11:23:39.719034 23637 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:23:39.716214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51414 (local address 127.16.197.1:34213)
0807 11:23:39.716868 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:39.716875 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:39.716904 (+ 29us) server_negotiation.cc:407] Connection header received
0807 11:23:39.717087 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:39.717094 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:39.717355 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:39.717555 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:39.718569 (+ 1014us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51414: BlockingRecv error: recv got EOF from 127.0.0.1:51414 (error 108)
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":308,"threads_started":1}
W20260807 11:23:39.719317 17172 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/krb5cc' of type FILE
Aug 07 11:23:39 dist-test-slave-2w3w krb5kdc[23097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:39.721328 17172 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 11:23:39 dist-test-slave-2w3w krb5kdc[23097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:39.722805 17172 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 11:23:39 dist-test-slave-2w3w krb5kdc[23097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.1@KRBTEST.COM
I20260807 11:23:39.739184 23637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:39.724790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51426 (local address 127.16.197.1:34213)
0807 11:23:39.725020 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:39.725026 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:39.725128 (+ 102us) server_negotiation.cc:407] Connection header received
0807 11:23:39.725319 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:39.725323 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:39.725388 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:39.725467 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:39.726487 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:39.727581 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:39.728984 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:39.729590 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:39.729695 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:39.733432 (+ 3737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:39.733447 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:39.733464 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:39.733501 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:39.736542 (+ 3041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:39.737232 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:39.737239 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:39.737241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:39.737376 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:39.737688 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:39.737691 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:39.737693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:39.738047 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:39.738281 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:39.738533 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 11:23:39.738796 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260807 11:23:39.745302 23637 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:23:39.743410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51436 (local address 127.16.197.1:34213)
0807 11:23:39.743571 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:39.743576 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:39.744052 (+ 476us) server_negotiation.cc:407] Connection header received
0807 11:23:39.744141 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:39.744144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:39.744204 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:39.744290 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:39.744997 (+ 707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51436: BlockingRecv error: recv got EOF from 127.0.0.1:51436 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 11:23:39.745512 17172 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/krb5cc' of type FILE
Aug 07 11:23:39 dist-test-slave-2w3w krb5kdc[23097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:39.747151 17172 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 11:23:39 dist-test-slave-2w3w krb5kdc[23097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:39.748641 17172 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 11:23:39 dist-test-slave-2w3w krb5kdc[23097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.1@KRBTEST.COM
I20260807 11:23:39.763193 23637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:39.750141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51442 (local address 127.16.197.1:34213)
0807 11:23:39.750342 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:39.750348 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:39.750526 (+ 178us) server_negotiation.cc:407] Connection header received
0807 11:23:39.750720 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:39.750731 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:39.750795 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:39.750899 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:39.751928 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:39.752655 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:39.753878 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:39.754319 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:39.754482 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:39.758398 (+ 3916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:39.758412 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:39.758415 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:39.758465 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:39.760920 (+ 2455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:39.761607 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:39.761611 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:39.761614 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:39.761733 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:39.762060 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:39.762063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:39.762065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:39.762320 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:39.762525 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:39.762738 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 11:23:39.762900 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260807 11:23:39.767891 23637 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:23:39.765901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51458 (local address 127.16.197.1:34213)
0807 11:23:39.766114 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:39.766122 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:39.766462 (+ 340us) server_negotiation.cc:407] Connection header received
0807 11:23:39.766651 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:39.766656 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:39.766735 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:39.766833 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:39.767640 (+ 807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51458: BlockingRecv error: recv got EOF from 127.0.0.1:51458 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260807 11:23:39.768190 17172 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786101765888975-17172-0/krb5kdc/krb5cc' of type FILE
Aug 07 11:23:39 dist-test-slave-2w3w krb5kdc[23097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:39.769805 17172 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 11:23:39 dist-test-slave-2w3w krb5kdc[23097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:39.771198 17172 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 11:23:39 dist-test-slave-2w3w krb5kdc[23097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.1@KRBTEST.COM
I20260807 11:23:39.785074 23637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:39.772557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51468 (local address 127.16.197.1:34213)
0807 11:23:39.772772 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:39.772779 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:39.772961 (+ 182us) server_negotiation.cc:407] Connection header received
0807 11:23:39.773094 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:39.773097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:39.773157 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:39.773235 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:39.774135 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:39.774864 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:39.776120 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:39.776544 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:39.776636 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:39.780210 (+ 3574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:39.780215 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:39.780219 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:39.780250 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:39.782624 (+ 2374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:39.783281 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:39.783284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:39.783287 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:39.783448 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:39.783814 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:39.783817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:39.783819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:39.784077 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:39.784308 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:39.784541 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 11:23:39.784751 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 11:23:39.786782 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 23207
I20260807 11:23:39.799083 23637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:39.787588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51478 (local address 127.16.197.1:34213)
0807 11:23:39.787741 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:39.787745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:39.788208 (+ 463us) server_negotiation.cc:407] Connection header received
0807 11:23:39.788350 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:39.788354 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:39.788410 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:39.788480 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:39.789440 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:39.790118 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:39.791291 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:39.791683 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:39.791784 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:39.793782 (+ 1998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:39.793787 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:39.793790 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:39.793820 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:39.796773 (+ 2953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:39.797368 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:39.797371 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:39.797374 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:39.797510 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:39.797808 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:39.797811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:39.797813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:39.798085 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:39.798305 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:39.798559 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 11:23:39.798731 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 11:23:39.799711 23338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:39.885423 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 23207
I20260807 11:23:39.903683 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 23350
Aug 07 11:23:39 dist-test-slave-2w3w krb5kdc[23097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.2@KRBTEST.COM
I20260807 11:23:39.923697 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:39.904669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51308 (local address 127.16.197.2:37145)
0807 11:23:39.905324 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:39.905332 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:39.905358 (+ 26us) server_negotiation.cc:407] Connection header received
0807 11:23:39.905426 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:39.905432 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:39.905642 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:39.905771 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:39.906758 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:39.907845 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:39.909120 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:39.909684 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:39.909790 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:39.917506 (+ 7716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:39.917525 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:39.917542 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:39.917588 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:39.920704 (+ 3116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:39.921517 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:39.921525 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:39.921528 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:39.921657 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:39.922017 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:39.922021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:39.922023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:39.922385 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:39.922628 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:39.922930 (+ 302us) server_negotiation.cc:299] Negotiation successful
0807 11:23:39.923374 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":361,"threads_started":1}
I20260807 11:23:39.925093 23478 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:40.011358 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 23350
I20260807 11:23:40.031087 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 23492
Aug 07 11:23:40 dist-test-slave-2w3w krb5kdc[23097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.3@KRBTEST.COM
I20260807 11:23:40.048612 23653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:40.032096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42352 (local address 127.16.197.3:42585)
0807 11:23:40.032797 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:40.032805 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:40.032835 (+ 30us) server_negotiation.cc:407] Connection header received
0807 11:23:40.032895 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:40.032901 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:40.033129 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:40.033304 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:40.034472 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:40.035723 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:40.037013 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:40.037608 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:40.037772 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:40.042341 (+ 4569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:40.042355 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:40.042371 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:40.042409 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:40.045573 (+ 3164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:40.046355 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:40.046364 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:40.046367 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:40.046516 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:40.046880 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:40.046884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:40.046886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:40.047363 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:40.047649 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:40.047935 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 11:23:40.048259 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":337,"threads_started":1}
I20260807 11:23:40.050078 23622 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:40.134397 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 23492
I20260807 11:23:40.151815 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 23111
I20260807 11:23:40.153074 23174 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:40.260262 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 23111
2026-08-07T11:23:40Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4116 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:23:40 dist-test-slave-2w3w krb5kdc[23658](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:23:40 dist-test-slave-2w3w krb5kdc[23658](info): set up 2 sockets
Aug 07 11:23:40 dist-test-slave-2w3w krb5kdc[23658](info): commencing operation
krb5kdc: starting...
W20260807 11:23:41.397003 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:23:41 dist-test-slave-2w3w krb5kdc[23658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:23:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:23:41Z Disabled control of system clock
WARNING: no policy specified for oryx/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.62.keytab.
Entry for principal oryx/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.62.keytab.
I20260807 11:23:41.639667 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:33957
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.62.keytab
--principal=oryx/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:44033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:33957
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:41.992197 23672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:41.992522 23672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:41.992708 23672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:42.000042 23672 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:23:42.000177 23672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:42.000295 23672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:42.000411 23672 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:23:42.000499 23672 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:23:42.011034 23672 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:44033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:33957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.62.keytab
--principal=oryx/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:33957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:42.012476 23672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:42.013854 23672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:42.026034 23677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:42.025943 23680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:42.025991 23678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:42.026840 23672 server_base.cc:1034] running on GCE node
I20260807 11:23:42.027684 23672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:42.029158 23672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:42.030366 23672 hybrid_clock.cc:648] HybridClock initialized: now 1786101822030316 us; error 41 us; skew 500 ppm
I20260807 11:23:42.030745 23672 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:42 dist-test-slave-2w3w krb5kdc[23658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101822, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:42.034003 23672 init.cc:377] Logged in from keytab as oryx/127.16.197.62@KRBTEST.COM (short username oryx)
I20260807 11:23:42.035672 23672 webserver.cc:460] Webserver started at http://127.16.197.62:34777/ using document root <none> and password file <none>
I20260807 11:23:42.036451 23672 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:42.036602 23672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:42.036929 23672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:42.040086 23672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "3f333150234643279ab25bac55bc7419"
format_stamp: "Formatted at 2026-08-07 11:23:42 on dist-test-slave-2w3w"
server_key: "0088d98d21e9a30598c60d52ec5f5ee7"
server_key_iv: "7253fa7994b4ea90eee2768bb657e89f"
server_key_version: "encryptionkey@0"
I20260807 11:23:42.040968 23672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "3f333150234643279ab25bac55bc7419"
format_stamp: "Formatted at 2026-08-07 11:23:42 on dist-test-slave-2w3w"
server_key: "0088d98d21e9a30598c60d52ec5f5ee7"
server_key_iv: "7253fa7994b4ea90eee2768bb657e89f"
server_key_version: "encryptionkey@0"
I20260807 11:23:42.046448 23672 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 11:23:42.050071 23687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:42.051501 23672 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:23:42.051692 23672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "3f333150234643279ab25bac55bc7419"
format_stamp: "Formatted at 2026-08-07 11:23:42 on dist-test-slave-2w3w"
server_key: "0088d98d21e9a30598c60d52ec5f5ee7"
server_key_iv: "7253fa7994b4ea90eee2768bb657e89f"
server_key_version: "encryptionkey@0"
I20260807 11:23:42.051908 23672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:42.075796 23672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:42.080233 23672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:42.080555 23672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:42.100078 23672 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:33957
I20260807 11:23:42.100126 23743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:33957 every 8 connection(s)
I20260807 11:23:42.101531 23672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:23:42.104261 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 23672
I20260807 11:23:42.104490 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:23:42.105383 17172 external_mini_cluster.cc:1428] Setting key 2aa2f3a70bc3892fb2ec2778c67574cd
I20260807 11:23:42.106133 23744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:42.121088 23744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419: Bootstrap starting.
Aug 07 11:23:42 dist-test-slave-2w3w krb5kdc[23658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM
I20260807 11:23:42.124305 23744 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:42.125491 23744 log.cc:824] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:42.128537 23744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419: No bootstrap required, opened a new log
I20260807 11:23:42.129631 23747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:42.107579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58594 (local address 127.16.197.62:33957)
0807 11:23:42.108391 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:42.108408 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:42.108451 (+ 43us) server_negotiation.cc:407] Connection header received
0807 11:23:42.110831 (+ 2380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:42.110854 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:42.111339 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:42.111692 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:42.113580 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:42.114905 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:42.116732 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:42.117454 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:42.117595 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:42.122277 (+ 4682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:42.122286 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:42.122300 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:42.122339 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:42.125511 (+ 3172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:42.126220 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:42.126227 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:42.126233 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:42.126414 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:42.126821 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:42.126826 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:42.126829 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:42.127384 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:42.127714 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:42.128156 (+ 442us) server_negotiation.cc:299] Negotiation successful
0807 11:23:42.128595 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":394,"threads_started":1}
I20260807 11:23:42.136844 23744 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } }
I20260807 11:23:42.137260 23744 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:42.137388 23744 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f333150234643279ab25bac55bc7419, State: Initialized, Role: FOLLOWER
I20260807 11:23:42.137966 23744 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } }
I20260807 11:23:42.138221 23744 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:23:42.138388 23744 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:23:42.138558 23744 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:42.141067 23744 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } }
I20260807 11:23:42.141652 23744 leader_election.cc:302] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f333150234643279ab25bac55bc7419; no voters:
I20260807 11:23:42.142354 23744 leader_election.cc:288] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:23:42.142500 23750 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:42.143314 23750 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [term 1 LEADER]: Becoming Leader. State: Replica: 3f333150234643279ab25bac55bc7419, State: Running, Role: LEADER
I20260807 11:23:42.143929 23750 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } }
I20260807 11:23:42.144735 23744 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:23:42.147445 23751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f333150234643279ab25bac55bc7419" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } }
I20260807 11:23:42.147639 23752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f333150234643279ab25bac55bc7419. Latest consensus state: current_term: 1 leader_uuid: "3f333150234643279ab25bac55bc7419" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } }
I20260807 11:23:42.147900 23751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:42.148011 23752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:42.149488 23757 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:23:42.156160 23757 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:23:42.164631 23757 catalog_manager.cc:1347] Generated new cluster ID: 9ded344e6a0f4ebf8ffae9aeddeaab78
I20260807 11:23:42.164763 23757 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:23:42.172245 23757 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:23:42.173553 23757 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:23:42.184449 23757 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419: Generated new TSK 0
I20260807 11:23:42.185395 23757 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.16.197.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.197.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.197.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.1.keytab.
Entry for principal oryx/127.16.197.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.1.keytab.
I20260807 11:23:42.243198 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:0
--local_ip_for_outbound_sockets=127.16.197.1
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:33957
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.1.keytab
--principal=oryx/127.16.197.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:44033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:42.580606 23771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:42.580910 23771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:42.581068 23771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:42.588379 23771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:42.588483 23771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:42.588634 23771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:23:42.599025 23771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:44033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.1.keytab
--principal=oryx/127.16.197.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:33957
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:42.600369 23771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:42.601766 23771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:42.615645 23776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:42.615687 23779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:42.615724 23777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:42.616849 23771 server_base.cc:1034] running on GCE node
I20260807 11:23:42.617506 23771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:42.618527 23771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:42.619726 23771 hybrid_clock.cc:648] HybridClock initialized: now 1786101822619670 us; error 48 us; skew 500 ppm
I20260807 11:23:42.620069 23771 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:42 dist-test-slave-2w3w krb5kdc[23658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101822, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:42.623389 23771 init.cc:377] Logged in from keytab as oryx/127.16.197.1@KRBTEST.COM (short username oryx)
I20260807 11:23:42.624876 23771 webserver.cc:460] Webserver started at http://127.16.197.1:42759/ using document root <none> and password file <none>
I20260807 11:23:42.625600 23771 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:42.625752 23771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:42.626072 23771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:42.629263 23771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/data/instance:
uuid: "fcdd78cc9b004378bf2531f78acafc78"
format_stamp: "Formatted at 2026-08-07 11:23:42 on dist-test-slave-2w3w"
server_key: "6332941e3b636e60fb5fd190e11b8cb7"
server_key_iv: "4c2f1cf8ee4a9de70fdb328f6b8f4e09"
server_key_version: "encryptionkey@0"
I20260807 11:23:42.630100 23771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance:
uuid: "fcdd78cc9b004378bf2531f78acafc78"
format_stamp: "Formatted at 2026-08-07 11:23:42 on dist-test-slave-2w3w"
server_key: "6332941e3b636e60fb5fd190e11b8cb7"
server_key_iv: "4c2f1cf8ee4a9de70fdb328f6b8f4e09"
server_key_version: "encryptionkey@0"
I20260807 11:23:42.635605 23771 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 11:23:42.639374 23786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:42.640702 23771 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 11:23:42.640898 23771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "fcdd78cc9b004378bf2531f78acafc78"
format_stamp: "Formatted at 2026-08-07 11:23:42 on dist-test-slave-2w3w"
server_key: "6332941e3b636e60fb5fd190e11b8cb7"
server_key_iv: "4c2f1cf8ee4a9de70fdb328f6b8f4e09"
server_key_version: "encryptionkey@0"
I20260807 11:23:42.641111 23771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:42.660948 23771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:42.665040 23771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:42.665334 23771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:42.666333 23771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:23:42.667802 23771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:42.667907 23771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:42.668058 23771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:42.668130 23771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:42.702821 23771 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:38243
I20260807 11:23:42.702917 23908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:38243 every 8 connection(s)
I20260807 11:23:42.704233 23771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:23:42.709638 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 23771
I20260807 11:23:42.709875 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance
I20260807 11:23:42.710546 17172 external_mini_cluster.cc:1428] Setting key 4918be341149444ad175fbbacb31a69d
Aug 07 11:23:42 dist-test-slave-2w3w krb5kdc[23658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101822, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM
I20260807 11:23:42.727900 23911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:42.706950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:33561 (local address 127.16.197.62:33957)
0807 11:23:42.707496 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:42.707501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:42.708448 (+ 947us) server_negotiation.cc:407] Connection header received
0807 11:23:42.710496 (+ 2048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:42.710502 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:42.710599 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:42.710741 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:42.712733 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:42.713502 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:42.716203 (+ 2701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:42.716599 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:42.716705 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:42.721210 (+ 4505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:42.721215 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:42.721218 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:42.721254 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:42.724332 (+ 3078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:42.725358 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:42.725362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:42.725364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:42.725487 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:42.726019 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:42.726023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:42.726024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:42.726287 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:42.726406 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:42.727427 (+ 1021us) server_negotiation.cc:299] Negotiation successful
0807 11:23:42.727612 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":266,"threads_started":1}
I20260807 11:23:42.729557 23912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:42.707522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.16.197.1:33561)
0807 11:23:42.708260 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:42.708307 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:42.710194 (+ 1887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:42.711081 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:42.711095 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:42.711680 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:42.712556 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:42.712577 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:42.713677 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:42.713685 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:42.716033 (+ 2348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:42.716052 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:42.716857 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:42.716866 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:42.717216 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:42.718030 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:42.718072 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:42.720930 (+ 2858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:42.724504 (+ 3574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:42.724518 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:42.724540 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:42.725214 (+ 674us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:42.725615 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:42.725623 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:42.725629 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:42.725900 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:42.726424 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:42.726435 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:42.726932 (+ 497us) client_negotiation.cc:769] Sending connection context
0807 11:23:42.727336 (+ 404us) client_negotiation.cc:240] Negotiation successful
0807 11:23:42.727977 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":295,"threads_started":1}
I20260807 11:23:42.731247 23909 heartbeater.cc:347] Connected to a master server at 127.16.197.62:33957
I20260807 11:23:42.731576 23909 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:42.732364 23909 heartbeater.cc:511] Master 127.16.197.62:33957 requested a full tablet report, sending...
I20260807 11:23:42.734921 23705 ts_manager.cc:194] Registered new tserver with Master: fcdd78cc9b004378bf2531f78acafc78 (127.16.197.1:38243)
I20260807 11:23:42.737217 23705 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.197.1@KRBTEST.COM'} at 127.16.197.1:33561
WARNING: no policy specified for oryx/127.16.197.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.197.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.197.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.2.keytab.
Entry for principal oryx/127.16.197.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.2.keytab.
I20260807 11:23:42.770496 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:0
--local_ip_for_outbound_sockets=127.16.197.2
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:33957
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.2.keytab
--principal=oryx/127.16.197.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:44033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:43.110683 23916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:43.111073 23916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:43.111256 23916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:43.118647 23916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:43.118829 23916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:43.119066 23916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:23:43.128978 23916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:44033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.2.keytab
--principal=oryx/127.16.197.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:33957
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:43.130316 23916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:43.131728 23916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:43.145150 23924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:43.145252 23921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:43.145289 23922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:43.146294 23916 server_base.cc:1034] running on GCE node
I20260807 11:23:43.147043 23916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:43.148025 23916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:43.149262 23916 hybrid_clock.cc:648] HybridClock initialized: now 1786101823149217 us; error 62 us; skew 500 ppm
I20260807 11:23:43.149619 23916 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:43 dist-test-slave-2w3w krb5kdc[23658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101823, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:43.152916 23916 init.cc:377] Logged in from keytab as oryx/127.16.197.2@KRBTEST.COM (short username oryx)
I20260807 11:23:43.154377 23916 webserver.cc:460] Webserver started at http://127.16.197.2:38979/ using document root <none> and password file <none>
I20260807 11:23:43.155169 23916 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:43.155328 23916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:43.155665 23916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:43.158720 23916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/data/instance:
uuid: "af5f0cfe2e664d478c15b24cb6071baf"
format_stamp: "Formatted at 2026-08-07 11:23:43 on dist-test-slave-2w3w"
server_key: "9bec3306606d2606a9fef95f88df335e"
server_key_iv: "237e4031a16f23497e70993eeb4167db"
server_key_version: "encryptionkey@0"
I20260807 11:23:43.159699 23916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance:
uuid: "af5f0cfe2e664d478c15b24cb6071baf"
format_stamp: "Formatted at 2026-08-07 11:23:43 on dist-test-slave-2w3w"
server_key: "9bec3306606d2606a9fef95f88df335e"
server_key_iv: "237e4031a16f23497e70993eeb4167db"
server_key_version: "encryptionkey@0"
I20260807 11:23:43.165339 23916 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 11:23:43.169056 23931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:43.170490 23916 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260807 11:23:43.170686 23916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "af5f0cfe2e664d478c15b24cb6071baf"
format_stamp: "Formatted at 2026-08-07 11:23:43 on dist-test-slave-2w3w"
server_key: "9bec3306606d2606a9fef95f88df335e"
server_key_iv: "237e4031a16f23497e70993eeb4167db"
server_key_version: "encryptionkey@0"
I20260807 11:23:43.170920 23916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:43.193216 23916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:43.197834 23916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:43.198148 23916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:43.199174 23916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:23:43.200569 23916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:43.200680 23916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:43.200847 23916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:43.200922 23916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:43.235622 23916 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:41871
I20260807 11:23:43.235714 24051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:41871 every 8 connection(s)
I20260807 11:23:43.236976 23916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:23:43.244910 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 23916
I20260807 11:23:43.245119 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance
I20260807 11:23:43.245688 17172 external_mini_cluster.cc:1428] Setting key b1c6192c4a470c2c83d4d375a2f51974
Aug 07 11:23:43 dist-test-slave-2w3w krb5kdc[23658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101823, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM
I20260807 11:23:43.259981 24055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:43.239815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:59413 (local address 127.16.197.62:33957)
0807 11:23:43.240279 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:43.240285 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:43.241332 (+ 1047us) server_negotiation.cc:407] Connection header received
0807 11:23:43.243343 (+ 2011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:43.243350 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:43.243446 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:43.243574 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:43.245646 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:43.246825 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:43.249060 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:43.249484 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:43.249627 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:43.253983 (+ 4356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:43.253988 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:43.253991 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:43.254024 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:43.256847 (+ 2823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:43.257597 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:43.257601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:43.257603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:43.257738 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:43.258179 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:43.258183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:43.258185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:43.258451 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:43.258610 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:43.259366 (+ 756us) server_negotiation.cc:299] Negotiation successful
0807 11:23:43.259588 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":228,"threads_started":1}
I20260807 11:23:43.260954 24056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:43.240394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.16.197.2:59413)
0807 11:23:43.241148 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:43.241194 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:43.242924 (+ 1730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:43.243843 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:43.243853 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:43.244493 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:43.245446 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:43.245474 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:43.247036 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:43.247040 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:43.248904 (+ 1864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:43.248922 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:43.249756 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:43.249774 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:43.250115 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:43.250982 (+ 867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:43.251021 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:43.253741 (+ 2720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:43.256991 (+ 3250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:43.257000 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:43.257018 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:43.257459 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:43.257852 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:43.257857 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:43.257861 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:43.258051 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:43.258580 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:43.258588 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:43.258937 (+ 349us) client_negotiation.cc:769] Sending connection context
0807 11:23:43.259267 (+ 330us) client_negotiation.cc:240] Negotiation successful
0807 11:23:43.259815 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":301,"threads_started":1}
I20260807 11:23:43.262497 24053 heartbeater.cc:347] Connected to a master server at 127.16.197.62:33957
I20260807 11:23:43.262830 24053 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:43.263780 24053 heartbeater.cc:511] Master 127.16.197.62:33957 requested a full tablet report, sending...
I20260807 11:23:43.265486 23705 ts_manager.cc:194] Registered new tserver with Master: af5f0cfe2e664d478c15b24cb6071baf (127.16.197.2:41871)
I20260807 11:23:43.266770 23705 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.197.2@KRBTEST.COM'} at 127.16.197.2:59413
WARNING: no policy specified for oryx/127.16.197.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.197.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.197.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.3.keytab.
Entry for principal oryx/127.16.197.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.3.keytab.
I20260807 11:23:43.301926 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:0
--local_ip_for_outbound_sockets=127.16.197.3
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:33957
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.3.keytab
--principal=oryx/127.16.197.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:44033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:43.632169 24059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:43.632436 24059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:43.632547 24059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:43.639848 24059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:43.639957 24059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:43.640069 24059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:23:43.650445 24059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:44033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.3.keytab
--principal=oryx/127.16.197.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:33957
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:43.652016 24059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:43.653445 24059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:43.667230 24067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:43.667258 24065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:43.667407 24064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:43.668169 24059 server_base.cc:1034] running on GCE node
I20260807 11:23:43.668913 24059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:43.669895 24059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:43.671074 24059 hybrid_clock.cc:648] HybridClock initialized: now 1786101823671023 us; error 37 us; skew 500 ppm
I20260807 11:23:43.671413 24059 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:43 dist-test-slave-2w3w krb5kdc[23658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101823, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:43.675060 24059 init.cc:377] Logged in from keytab as oryx/127.16.197.3@KRBTEST.COM (short username oryx)
I20260807 11:23:43.676661 24059 webserver.cc:460] Webserver started at http://127.16.197.3:33565/ using document root <none> and password file <none>
I20260807 11:23:43.677397 24059 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:43.677529 24059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:43.677865 24059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:43.680929 24059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/data/instance:
uuid: "2fd50e45dcc24d218e3c7f4951e5bf86"
format_stamp: "Formatted at 2026-08-07 11:23:43 on dist-test-slave-2w3w"
server_key: "cf11bcc2b12a78399ed70de655d629cc"
server_key_iv: "c400b3e2e74ad8842f2bf72602a52496"
server_key_version: "encryptionkey@0"
I20260807 11:23:43.681762 24059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance:
uuid: "2fd50e45dcc24d218e3c7f4951e5bf86"
format_stamp: "Formatted at 2026-08-07 11:23:43 on dist-test-slave-2w3w"
server_key: "cf11bcc2b12a78399ed70de655d629cc"
server_key_iv: "c400b3e2e74ad8842f2bf72602a52496"
server_key_version: "encryptionkey@0"
I20260807 11:23:43.688254 24059 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260807 11:23:43.692203 24074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:43.693892 24059 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260807 11:23:43.694082 24059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "2fd50e45dcc24d218e3c7f4951e5bf86"
format_stamp: "Formatted at 2026-08-07 11:23:43 on dist-test-slave-2w3w"
server_key: "cf11bcc2b12a78399ed70de655d629cc"
server_key_iv: "c400b3e2e74ad8842f2bf72602a52496"
server_key_version: "encryptionkey@0"
I20260807 11:23:43.694295 24059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:43.720234 24059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:43.724964 24059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:43.725299 24059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:43.726311 24059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:23:43.727792 24059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:43.727893 24059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:43.728034 24059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:43.728104 24059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:43.740617 23909 heartbeater.cc:503] Master 127.16.197.62:33957 was elected leader, sending a full tablet report...
I20260807 11:23:43.772135 24059 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:39535
I20260807 11:23:43.772248 24196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:39535 every 8 connection(s)
I20260807 11:23:43.773530 24059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:23:43.776098 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 24059
I20260807 11:23:43.776301 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance
I20260807 11:23:43.776778 17172 external_mini_cluster.cc:1428] Setting key e53b96e89b005213b4fd27cc7ffc03e6
Aug 07 11:23:43 dist-test-slave-2w3w krb5kdc[23658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101823, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM
I20260807 11:23:43.798149 24199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:43.776995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:46123 (local address 127.16.197.62:33957)
0807 11:23:43.777523 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:43.777530 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:43.778680 (+ 1150us) server_negotiation.cc:407] Connection header received
0807 11:23:43.780839 (+ 2159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:43.780845 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:43.780946 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:43.781067 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:43.782483 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:43.783848 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:43.785408 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:43.786176 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:43.786322 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:43.790024 (+ 3702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:43.790032 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:43.790038 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:43.790088 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:43.794494 (+ 4406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:43.795349 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:43.795359 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:43.795365 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:43.795536 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:43.795992 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:43.795999 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:43.796005 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:43.796410 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:43.796594 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:43.797293 (+ 699us) server_negotiation.cc:299] Negotiation successful
0807 11:23:43.797594 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":238,"threads_started":1}
I20260807 11:23:43.799098 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:43.777631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.16.197.3:46123)
0807 11:23:43.778507 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:43.778558 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:43.780574 (+ 2016us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:43.781233 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:43.781241 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:43.781739 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:43.782307 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:43.782324 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:43.784015 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:43.784019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:43.785251 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:43.785269 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:43.786387 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:43.786392 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:43.786582 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:43.787343 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:43.787378 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:43.789764 (+ 2386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:43.794644 (+ 4880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:43.794652 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:43.794667 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:43.795227 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:43.795659 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:43.795662 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:43.795664 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:43.795851 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:43.796544 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:43.796549 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:43.796891 (+ 342us) client_negotiation.cc:769] Sending connection context
0807 11:23:43.797124 (+ 233us) client_negotiation.cc:240] Negotiation successful
0807 11:23:43.797573 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":329,"threads_started":1}
I20260807 11:23:43.800556 24197 heartbeater.cc:347] Connected to a master server at 127.16.197.62:33957
I20260807 11:23:43.800935 24197 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:43.802033 24197 heartbeater.cc:511] Master 127.16.197.62:33957 requested a full tablet report, sending...
I20260807 11:23:43.803960 23705 ts_manager.cc:194] Registered new tserver with Master: 2fd50e45dcc24d218e3c7f4951e5bf86 (127.16.197.3:39535)
I20260807 11:23:43.804838 23705 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.197.3@KRBTEST.COM'} at 127.16.197.3:46123
I20260807 11:23:43.805606 17172 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.16.197.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.197.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.197.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.61.keytab.
Entry for principal oryx/127.16.197.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.61.keytab.
I20260807 11:23:43.859894 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.61:41937
--webserver_interface=127.16.197.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.61.keytab
--principal=oryx/127.16.197.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:44033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:33957,127.16.197.61:41937
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:44.196094 24203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:44.196389 24203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:44.196480 24203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:44.203640 24203 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:23:44.203758 24203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:44.203805 24203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:44.203862 24203 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:23:44.203907 24203 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:23:44.213414 24203 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:44033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:33957,127.16.197.61:41937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.61.keytab
--principal=oryx/127.16.197.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.61:41937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.16.197.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
W20260807 11:23:44.213711 24203 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.16.197.62:33957,127.16.197.61:41937'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 11:23:44.214692 24203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:44.216015 24203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:44.227627 24208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:44.227713 24210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:44.227896 24212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:44.228649 24203 server_base.cc:1034] running on GCE node
I20260807 11:23:44.229346 24203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:44.230643 24203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:44.231853 24203 hybrid_clock.cc:648] HybridClock initialized: now 1786101824231767 us; error 61 us; skew 500 ppm
I20260807 11:23:44.232187 24203 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:44 dist-test-slave-2w3w krb5kdc[23658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101824, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:44.235638 24203 init.cc:377] Logged in from keytab as oryx/127.16.197.61@KRBTEST.COM (short username oryx)
I20260807 11:23:44.237310 24203 webserver.cc:460] Webserver started at http://127.16.197.61:37481/ using document root <none> and password file <none>
I20260807 11:23:44.238024 24203 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:44.238147 24203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:44.238516 24203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:44.241511 24203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-1/data/instance:
uuid: "b2544d96e5a14f06b63cc24304065247"
format_stamp: "Formatted at 2026-08-07 11:23:44 on dist-test-slave-2w3w"
server_key: "36503cb3275e96d2c69759740416dab1"
server_key_iv: "95bb4a399fb7d0d6497abcce36bfcab5"
server_key_version: "encryptionkey@0"
I20260807 11:23:44.242295 24203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-1/wal/instance:
uuid: "b2544d96e5a14f06b63cc24304065247"
format_stamp: "Formatted at 2026-08-07 11:23:44 on dist-test-slave-2w3w"
server_key: "36503cb3275e96d2c69759740416dab1"
server_key_iv: "95bb4a399fb7d0d6497abcce36bfcab5"
server_key_version: "encryptionkey@0"
I20260807 11:23:44.247746 24203 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 11:23:44.251194 24219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:44.252561 24203 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 11:23:44.252743 24203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-1/wal
uuid: "b2544d96e5a14f06b63cc24304065247"
format_stamp: "Formatted at 2026-08-07 11:23:44 on dist-test-slave-2w3w"
server_key: "36503cb3275e96d2c69759740416dab1"
server_key_iv: "95bb4a399fb7d0d6497abcce36bfcab5"
server_key_version: "encryptionkey@0"
I20260807 11:23:44.252993 24203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:44.270354 24053 heartbeater.cc:503] Master 127.16.197.62:33957 was elected leader, sending a full tablet report...
I20260807 11:23:44.280925 24203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:44.284953 24203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:44.285175 24203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:44.303815 24203 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.61:41937
I20260807 11:23:44.303822 24273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.61:41937 every 8 connection(s)
I20260807 11:23:44.305125 24203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-1/data/info.pb
I20260807 11:23:44.308682 24274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:44.313912 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 24203
I20260807 11:23:44.315246 24274 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 11:23:44 dist-test-slave-2w3w krb5kdc[23658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101824, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.61@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM
Aug 07 11:23:44 dist-test-slave-2w3w krb5kdc[23658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.197.61@KRBTEST.COM
I20260807 11:23:44.335573 24278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:44.318127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58610 (local address 127.16.197.62:33957)
0807 11:23:44.318634 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:44.318642 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:44.319190 (+ 548us) server_negotiation.cc:407] Connection header received
0807 11:23:44.319619 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:44.319625 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:44.319739 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:44.319853 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:44.321101 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.321916 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.323678 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.324214 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.324309 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:44.328403 (+ 4094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:44.328410 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:44.328415 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:44.328463 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:44.331559 (+ 3096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:44.332650 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:44.332655 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:44.332657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:44.332817 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:44.333348 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:44.333354 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:44.333357 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:44.333823 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:44.333998 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:44.334902 (+ 904us) server_negotiation.cc:299] Negotiation successful
0807 11:23:44.335192 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":266,"threads_started":1}
I20260807 11:23:44.336900 24277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:44.316092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47468 (local address 127.16.197.61:41937)
0807 11:23:44.316886 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:44.316896 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:44.316936 (+ 40us) server_negotiation.cc:407] Connection header received
0807 11:23:44.319298 (+ 2362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:44.319330 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:44.319551 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:44.319728 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:44.321188 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.322576 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.323965 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.324734 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.324890 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:44.329390 (+ 4500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:44.329403 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:44.329421 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:44.329461 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:44.332895 (+ 3434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:44.333697 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:44.333704 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:44.333706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:44.333820 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:44.334242 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:44.334247 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:44.334250 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:44.334661 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:44.335009 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:44.335338 (+ 329us) server_negotiation.cc:299] Negotiation successful
0807 11:23:44.335550 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":102,"server-negotiator.queue_time_us":596,"thread_start_us":401,"threads_started":1}
I20260807 11:23:44.336925 24279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:44.318157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.0.0.1:58610)
0807 11:23:44.319050 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:44.319084 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:44.319391 (+ 307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:44.319944 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:44.319951 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:44.320591 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:44.320927 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:44.320943 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.322065 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.322070 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:44.323537 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:44.323549 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.324429 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.324436 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:44.324778 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:44.325724 (+ 946us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:44.325744 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:44.328172 (+ 2428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:44.331704 (+ 3532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:44.331714 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:44.331726 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:44.332506 (+ 780us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:44.332928 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:44.332935 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:44.332942 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:44.333212 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:44.333973 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:44.333987 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:44.334396 (+ 409us) client_negotiation.cc:769] Sending connection context
0807 11:23:44.334762 (+ 366us) client_negotiation.cc:240] Negotiation successful
0807 11:23:44.335328 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"mutex_wait_us":90,"spinlock_wait_cycles":33408,"thread_start_us":480,"threads_started":1}
I20260807 11:23:44.339193 24274 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 11:23:44 dist-test-slave-2w3w krb5kdc[23658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101824, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.61@KRBTEST.COM for oryx/127.16.197.61@KRBTEST.COM
I20260807 11:23:44.351624 24279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:44.340060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:41937 (local address 127.0.0.1:47484)
0807 11:23:44.340195 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:44.340210 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:44.340298 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:44.340534 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:44.340537 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:44.340851 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:44.340993 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:44.341001 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.342075 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.342078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:44.342948 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:44.343015 (+ 67us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.343687 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.343691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:44.343828 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:44.344347 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:44.344363 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:44.346547 (+ 2184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:44.349374 (+ 2827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:44.349377 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:44.349380 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:44.349819 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:44.350164 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:44.350167 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:44.350170 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:44.350300 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:44.350798 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:44.350801 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:44.350925 (+ 124us) client_negotiation.cc:769] Sending connection context
0807 11:23:44.351103 (+ 178us) client_negotiation.cc:240] Negotiation successful
0807 11:23:44.351318 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 11:23:44.351703 24277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:44.339959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47484 (local address 127.16.197.61:41937)
0807 11:23:44.340164 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:44.340169 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:44.340277 (+ 108us) server_negotiation.cc:407] Connection header received
0807 11:23:44.340371 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:44.340374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:44.340426 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:44.340500 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:44.341148 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.341951 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.343147 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.343552 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.343666 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:44.346713 (+ 3047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:44.346718 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:44.346721 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:44.346749 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:44.349190 (+ 2441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:44.349925 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:44.349929 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:44.349931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:44.350068 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:44.350414 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:44.350421 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:44.350424 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:44.350694 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:44.350821 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:44.351193 (+ 372us) server_negotiation.cc:299] Negotiation successful
0807 11:23:44.351452 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":75,"server-negotiator.queue_time_us":69}
I20260807 11:23:44.356761 24274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247: Bootstrap starting.
I20260807 11:23:44.359993 24274 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:44.360954 24274 log.cc:824] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:44.363511 24274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247: No bootstrap required, opened a new log
I20260807 11:23:44.370328 24274 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } }
I20260807 11:23:44.370630 24274 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:44.370700 24274 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2544d96e5a14f06b63cc24304065247, State: Initialized, Role: FOLLOWER
I20260807 11:23:44.371282 24274 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } }
I20260807 11:23:44.372067 24281 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } } }
I20260807 11:23:44.372367 24281 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 11:23:44.373030 24274 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 11:23:44.381611 24292 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 11:23:44.381750 24292 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 11:23:44.385409 24203 master_runner.cc:428] Detected that this master b2544d96e5a14f06b63cc24304065247 is joining an existing cluster
I20260807 11:23:44.385541 24203 master_runner.cc:432] Initiating AddMaster RPC to add 127.16.197.61:41937
I20260807 11:23:44.414275 24278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:44.389790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58614 (local address 127.16.197.62:33957)
0807 11:23:44.390010 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:44.390019 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:44.396679 (+ 6660us) server_negotiation.cc:407] Connection header received
0807 11:23:44.396996 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:44.397003 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:44.397100 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:44.397182 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:44.398893 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.400159 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.402547 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.403343 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.403463 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:44.406365 (+ 2902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:44.406372 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:44.406378 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:44.406422 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:44.410364 (+ 3942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:44.411458 (+ 1094us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:44.411465 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:44.411469 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:44.411644 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:44.412258 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:44.412264 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:44.412268 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:44.412695 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:44.412875 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:44.413583 (+ 708us) server_negotiation.cc:299] Negotiation successful
0807 11:23:44.413872 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 11:23:44.417083 24299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:44.390065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.0.0.1:58614)
0807 11:23:44.396561 (+ 6496us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:44.396585 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:44.396821 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:44.397438 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:44.397442 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:44.398222 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:44.398675 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:44.398691 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.400367 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.400373 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:44.402351 (+ 1978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:44.402368 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.403812 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.403821 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:44.404038 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:44.404778 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:44.404803 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:44.406106 (+ 1303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:44.410539 (+ 4433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:44.410544 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:44.410549 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:44.411274 (+ 725us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:44.411891 (+ 617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:44.411896 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:44.411900 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:44.412091 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:44.412984 (+ 893us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:44.412990 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:44.413213 (+ 223us) client_negotiation.cc:769] Sending connection context
0807 11:23:44.414602 (+ 1389us) client_negotiation.cc:240] Negotiation successful
0807 11:23:44.416523 (+ 1921us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6338,"thread_start_us":5608,"threads_started":1}
WARNING: no policy specified for oryx/127.16.197.60@KRBTEST.COM; defaulting to no policy
I20260807 11:23:44.421337 24277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:44.389829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47492 (local address 127.16.197.61:41937)
0807 11:23:44.390192 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:44.390204 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:44.401767 (+ 11563us) server_negotiation.cc:407] Connection header received
0807 11:23:44.402177 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:44.402184 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:44.402260 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:44.402324 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:44.403845 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.405437 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.408064 (+ 2627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.409175 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.409284 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:44.412070 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:44.412078 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:44.412085 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:44.412133 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:44.416780 (+ 4647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:44.418030 (+ 1250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:44.418035 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:44.418039 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:44.418220 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:44.418674 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:44.418679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:44.418683 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:44.419136 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:44.419288 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:44.419852 (+ 564us) server_negotiation.cc:299] Negotiation successful
0807 11:23:44.420087 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222}
I20260807 11:23:44.423501 24300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:44.390831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:41937 (local address 127.0.0.1:47492)
0807 11:23:44.401667 (+ 10836us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:44.401689 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:44.402035 (+ 346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:44.402537 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:44.402544 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:44.403294 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:44.403674 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:44.403690 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.405767 (+ 2077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.405775 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:44.407909 (+ 2134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:44.407926 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.409609 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.409617 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:44.409841 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:44.410520 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:44.410546 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:44.411884 (+ 1338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:44.416957 (+ 5073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:44.416962 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:44.416966 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:44.417894 (+ 928us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:44.418380 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:44.418385 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:44.418388 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:44.418553 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:44.419368 (+ 815us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:44.419373 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:44.419554 (+ 181us) client_negotiation.cc:769] Sending connection context
0807 11:23:44.422551 (+ 2997us) client_negotiation.cc:240] Negotiation successful
0807 11:23:44.422994 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10675,"thread_start_us":10897,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.197.60@KRBTEST.COM" created.
Aug 07 11:23:44 dist-test-slave-2w3w krb5kdc[23658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101822, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.62@KRBTEST.COM for oryx/127.16.197.61@KRBTEST.COM
I20260807 11:23:44.440757 24303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:44.424718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:41937 (local address 127.0.0.1:47498)
0807 11:23:44.425859 (+ 1141us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:44.425908 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:44.426094 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:44.426625 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:44.426632 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:44.427378 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:44.427598 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:44.427607 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.428629 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.428634 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:44.429623 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:44.429635 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.430376 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.430383 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:44.430571 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:44.431245 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:44.431262 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:44.433415 (+ 2153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:44.437226 (+ 3811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:44.437242 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:44.437253 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:44.437864 (+ 611us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:44.438327 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:44.438335 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:44.438342 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:44.438530 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:44.439303 (+ 773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:44.439314 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:44.439566 (+ 252us) client_negotiation.cc:769] Sending connection context
0807 11:23:44.439803 (+ 237us) client_negotiation.cc:240] Negotiation successful
0807 11:23:44.440190 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":883,"thread_start_us":681,"threads_started":1}
I20260807 11:23:44.441985 24277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:44.424685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47498 (local address 127.16.197.61:41937)
0807 11:23:44.424849 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:44.424856 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:44.426015 (+ 1159us) server_negotiation.cc:407] Connection header received
0807 11:23:44.426332 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:44.426340 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:44.426448 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:44.426565 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:44.427757 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.428494 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.429772 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.430211 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.430343 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:44.433609 (+ 3266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:44.433614 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:44.433617 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:44.433644 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:44.437062 (+ 3418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:44.438013 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:44.438022 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:44.438028 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:44.438187 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:44.438667 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:44.438674 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:44.438680 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:44.439083 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:44.439226 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:44.441225 (+ 1999us) server_negotiation.cc:299] Negotiation successful
0807 11:23:44.441523 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 11:23:44.445138 23704 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.16.197.61:41937: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b2544d96e5a14f06b63cc24304065247"
member_type: NON_VOTER
last_known_addr {
host: "127.16.197.61"
port: 41937
}
attrs {
promote: true
}
}
dest_uuid: "3f333150234643279ab25bac55bc7419"
cas_config_opid_index: -1
I20260807 11:23:44.446820 23704 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: NON_VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: true } }
I20260807 11:23:44.455397 24306 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER b2544d96e5a14f06b63cc24304065247 (127.16.197.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: NON_VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: true } } }
I20260807 11:23:44.456988 24248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:44.461809 24248 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [term 1 FOLLOWER]: Refusing update from remote peer 3f333150234643279ab25bac55bc7419: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 11:23:44.461791 24306 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.16.197.61:41937
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.197.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.60.keytab.
Entry for principal oryx/127.16.197.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.60.keytab.
I20260807 11:23:44.463186 24203 master.cc:537] Master@127.16.197.61:41937 shutting down...
I20260807 11:23:44.464151 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.60:32811
--webserver_interface=127.16.197.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.60.keytab
--principal=oryx/127.16.197.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:44033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:33957,127.16.197.61:41937,127.16.197.60:32811
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 11:23:44.482717 24307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3f333150234643279ab25bac55bc7419" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: NON_VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: true } } }
I20260807 11:23:44.483479 24307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:44.484294 24307 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: NON_VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:44.486258 24203 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:44.486894 24203 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:44.487226 24203 tablet_replica.cc:330] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247: stopping tablet replica
W20260807 11:23:44.489372 23688 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260807 11:23:44.495621 24203 master.cc:552] Master@127.16.197.61:41937 shutdown complete.
I20260807 11:23:44.495848 24203 master_runner.cc:305] Clearing existing system tablet
I20260807 11:23:44.496418 24203 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:44.496600 24203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
W20260807 11:23:44.497584 23688 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 -> Peer b2544d96e5a14f06b63cc24304065247 (127.16.197.61:41937): Couldn't send request to peer b2544d96e5a14f06b63cc24304065247. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260807 11:23:44.500095 24203 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 11:23:44.502044 24313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:44.502449 24203 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:44.502758 24203 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 11:23:44.502846 24203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-1/wal
uuid: "b2544d96e5a14f06b63cc24304065247"
format_stamp: "Formatted at 2026-08-07 11:23:44 on dist-test-slave-2w3w"
server_key: "36503cb3275e96d2c69759740416dab1"
server_key_iv: "95bb4a399fb7d0d6497abcce36bfcab5"
server_key_version: "encryptionkey@0"
I20260807 11:23:44.508759 24203 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:44.516145 24203 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 11:23:44.516340 24203 log.cc:1196] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 11:23:44.516821 24203 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247: Deleting consensus metadata
I20260807 11:23:44.517094 24203 master_runner.cc:315] Copying system tablet from 127.16.197.62:33957
I20260807 11:23:44.519855 24203 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247: tablet copy: Beginning tablet copy session from remote peer at address 127.16.197.62:33957
I20260807 11:23:44.541371 24319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:44.520855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.0.0.1:58624)
0807 11:23:44.526282 (+ 5427us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:44.526304 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:44.526485 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:44.526906 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:44.526911 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:44.527498 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:44.527778 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:44.527792 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.529289 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.529295 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:44.530819 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:44.530835 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.531875 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.531883 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:44.532070 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:44.532831 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:44.532856 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:44.533961 (+ 1105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:44.538160 (+ 4199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:44.538166 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:44.538170 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:44.538713 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:44.539208 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:44.539214 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:44.539218 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:44.539404 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:44.540159 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:44.540165 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:44.540372 (+ 207us) client_negotiation.cc:769] Sending connection context
0807 11:23:44.540534 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 11:23:44.540874 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5260,"thread_start_us":5150,"threads_started":1}
I20260807 11:23:44.542049 24278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:44.520919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58624 (local address 127.16.197.62:33957)
0807 11:23:44.521115 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:44.521121 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:44.526452 (+ 5331us) server_negotiation.cc:407] Connection header received
0807 11:23:44.526646 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:44.526652 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:44.526744 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:44.526864 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:44.527960 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.529114 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.531024 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.531698 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.531841 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:44.534189 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:44.534196 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:44.534201 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:44.534244 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:44.538015 (+ 3771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:44.538864 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:44.538869 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:44.538872 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:44.539068 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:44.539541 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:44.539548 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:44.539553 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:44.540000 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:44.540177 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:44.540680 (+ 503us) server_negotiation.cc:299] Negotiation successful
0807 11:23:44.541106 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 11:23:44.543103 23726 tablet_copy_service.cc:140] P 3f333150234643279ab25bac55bc7419: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b2544d96e5a14f06b63cc24304065247 ({username='oryx', principal='oryx/127.16.197.61@KRBTEST.COM'} at 127.0.0.1:58624)
I20260807 11:23:44.543512 23726 tablet_copy_service.cc:161] P 3f333150234643279ab25bac55bc7419: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b2544d96e5a14f06b63cc24304065247 at {username='oryx', principal='oryx/127.16.197.61@KRBTEST.COM'} at 127.0.0.1:58624: session id = b2544d96e5a14f06b63cc24304065247-00000000000000000000000000000000
I20260807 11:23:44.548648 23726 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 11:23:44.551508 24203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 11:23:44.558597 24203 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247: tablet copy: Starting download of 0 data blocks...
I20260807 11:23:44.559036 24203 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247: tablet copy: Starting download of 1 WAL segments...
I20260807 11:23:44.562532 24203 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 11:23:44.567263 23726 tablet_copy_service.cc:342] P 3f333150234643279ab25bac55bc7419: Request end of tablet copy session b2544d96e5a14f06b63cc24304065247-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.16.197.61@KRBTEST.COM'} at 127.0.0.1:58624
I20260807 11:23:44.567536 23726 tablet_copy_service.cc:433] P 3f333150234643279ab25bac55bc7419: ending tablet copy session b2544d96e5a14f06b63cc24304065247-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b2544d96e5a14f06b63cc24304065247
I20260807 11:23:44.580080 24203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:44.582866 24320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:44.586692 24321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:44.594388 24203 server_base.cc:1034] running on GCE node
W20260807 11:23:44.594576 24324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:44.595189 24203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:44.595930 24203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:44.597163 24203 hybrid_clock.cc:648] HybridClock initialized: now 1786101824597104 us; error 34 us; skew 500 ppm
I20260807 11:23:44.597415 24203 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:44 dist-test-slave-2w3w krb5kdc[23658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101824, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:44.599727 24203 init.cc:377] Logged in from keytab as oryx/127.16.197.61@KRBTEST.COM (short username oryx)
I20260807 11:23:44.605273 24203 webserver.cc:460] Webserver started at http://127.16.197.61:43973/ using document root <none> and password file <none>
I20260807 11:23:44.605679 24203 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:44.605784 24203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:44.608573 24203 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 11:23:44.610483 24332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:44.611057 24203 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 11:23:44.611229 24203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-1/wal
uuid: "b2544d96e5a14f06b63cc24304065247"
format_stamp: "Formatted at 2026-08-07 11:23:44 on dist-test-slave-2w3w"
server_key: "36503cb3275e96d2c69759740416dab1"
server_key_iv: "95bb4a399fb7d0d6497abcce36bfcab5"
server_key_version: "encryptionkey@0"
I20260807 11:23:44.611411 24203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:44.631661 24203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:44.635607 24203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:44.668270 24203 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.61:41937
I20260807 11:23:44.668318 24386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.61:41937 every 8 connection(s)
I20260807 11:23:44.669339 24203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-1/data/info.pb
I20260807 11:23:44.678020 24387 sys_catalog.cc:263] Verifying existing consensus state
I20260807 11:23:44.683916 24387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247: Bootstrap starting.
I20260807 11:23:44.699615 24387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 11:23:44.700167 24387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247: Bootstrap complete.
I20260807 11:23:44.701124 24387 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: NON_VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: true } }
I20260807 11:23:44.701378 24387 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b2544d96e5a14f06b63cc24304065247, State: Initialized, Role: LEARNER
I20260807 11:23:44.701632 24387 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: NON_VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: true } }
I20260807 11:23:44.702538 24390 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: NON_VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: true } } }
I20260807 11:23:44.702945 24387 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:23:44.703032 24390 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [sys.catalog]: This master's current role is: LEARNER
I20260807 11:23:44.711869 24402 catalog_manager.cc:1259] Loaded cluster ID: 9ded344e6a0f4ebf8ffae9aeddeaab78
I20260807 11:23:44.711972 24402 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247: loading cluster ID for follower catalog manager: success
I20260807 11:23:44.715294 24402 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247: acquiring CA information for follower catalog manager: success
I20260807 11:23:44.717746 24402 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 11:23:44.807863 24197 heartbeater.cc:503] Master 127.16.197.62:33957 was elected leader, sending a full tablet report...
I20260807 11:23:44.934679 24303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:44.921326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:41937 (local address 127.0.0.1:47500)
0807 11:23:44.921717 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:44.921743 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:44.921902 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:44.924154 (+ 2252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:44.924160 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:44.924219 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:44.924615 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:44.924640 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.927995 (+ 3355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.928000 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:44.930781 (+ 2781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:44.930805 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.933219 (+ 2414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.933227 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:44.933628 (+ 401us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:44.933754 (+ 126us) client_negotiation.cc:769] Sending connection context
0807 11:23:44.934023 (+ 269us) client_negotiation.cc:240] Negotiation successful
0807 11:23:44.934269 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185}
I20260807 11:23:44.935125 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:44.921630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47500 (local address 127.16.197.61:41937)
0807 11:23:44.923698 (+ 2068us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:44.923705 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:44.923729 (+ 24us) server_negotiation.cc:407] Connection header received
0807 11:23:44.923808 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:44.923814 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:44.923915 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:44.924075 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:44.924863 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.927704 (+ 2841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.931015 (+ 3311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:44.932940 (+ 1925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:44.933089 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:44.933489 (+ 400us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:44.934083 (+ 594us) server_negotiation.cc:299] Negotiation successful
0807 11:23:44.934316 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1885,"thread_start_us":1622,"threads_started":1}
I20260807 11:23:44.937502 24362 raft_consensus.cc:1203] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260807 11:23:44.938644 24390 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f333150234643279ab25bac55bc7419. Latest consensus state: current_term: 1 leader_uuid: "3f333150234643279ab25bac55bc7419" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: NON_VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: true } } }
I20260807 11:23:44.939075 24390 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [sys.catalog]: This master's current role is: LEARNER
W20260807 11:23:45.071062 24309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:45.071482 24309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:45.071676 24309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:45.081319 24309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:23:45.081468 24309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:45.081533 24309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:45.081620 24309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:23:45.081679 24309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:23:45.093746 24309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:44033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:33957,127.16.197.61:41937,127.16.197.60:32811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.60.keytab
--principal=oryx/127.16.197.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.60:32811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.16.197.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:45.095470 24309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:45.098344 24309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:45.123034 24409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:45.123121 24410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:45.123754 24412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:45.125490 24309 server_base.cc:1034] running on GCE node
I20260807 11:23:45.126362 24309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:45.128077 24309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:45.129352 24309 hybrid_clock.cc:648] HybridClock initialized: now 1786101825129266 us; error 44 us; skew 500 ppm
I20260807 11:23:45.129757 24309 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:45 dist-test-slave-2w3w krb5kdc[23658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101825, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:45.139227 24309 init.cc:377] Logged in from keytab as oryx/127.16.197.60@KRBTEST.COM (short username oryx)
I20260807 11:23:45.141762 24309 webserver.cc:460] Webserver started at http://127.16.197.60:37515/ using document root <none> and password file <none>
I20260807 11:23:45.142689 24309 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:45.142928 24309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:45.143443 24309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:45.147691 24309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-2/data/instance:
uuid: "d2d3da4008c443a6a7596fdac5b97711"
format_stamp: "Formatted at 2026-08-07 11:23:45 on dist-test-slave-2w3w"
server_key: "736dca78f02c46f10973b199386807a7"
server_key_iv: "4ca46d5a0bd47bbe85209e98e8ee7b14"
server_key_version: "encryptionkey@0"
I20260807 11:23:45.148671 24309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-2/wal/instance:
uuid: "d2d3da4008c443a6a7596fdac5b97711"
format_stamp: "Formatted at 2026-08-07 11:23:45 on dist-test-slave-2w3w"
server_key: "736dca78f02c46f10973b199386807a7"
server_key_iv: "4ca46d5a0bd47bbe85209e98e8ee7b14"
server_key_version: "encryptionkey@0"
I20260807 11:23:45.155866 24309 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260807 11:23:45.160888 24419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:45.162351 24309 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.001s
I20260807 11:23:45.162557 24309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-2/wal
uuid: "d2d3da4008c443a6a7596fdac5b97711"
format_stamp: "Formatted at 2026-08-07 11:23:45 on dist-test-slave-2w3w"
server_key: "736dca78f02c46f10973b199386807a7"
server_key_iv: "4ca46d5a0bd47bbe85209e98e8ee7b14"
server_key_version: "encryptionkey@0"
I20260807 11:23:45.162801 24309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:45.206725 24309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:45.213600 24309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:45.214123 24309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:45.264124 24309 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.60:32811
I20260807 11:23:45.264165 24475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.60:32811 every 8 connection(s)
I20260807 11:23:45.265760 24309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-2/data/info.pb
I20260807 11:23:45.268585 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 24309
I20260807 11:23:45.271626 24476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:45.285632 24476 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 11:23:45 dist-test-slave-2w3w krb5kdc[23658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.197.60@KRBTEST.COM
Aug 07 11:23:45 dist-test-slave-2w3w krb5kdc[23658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101825, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.60@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM
I20260807 11:23:45.304859 24479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:45.271494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46082 (local address 127.16.197.60:32811)
0807 11:23:45.273029 (+ 1535us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:45.273040 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:45.273115 (+ 75us) server_negotiation.cc:407] Connection header received
0807 11:23:45.276002 (+ 2887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:45.276026 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:45.276456 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:45.276809 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:45.279729 (+ 2920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.281459 (+ 1730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.283795 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.284647 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.284785 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:45.293244 (+ 8459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:45.293280 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:45.293299 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:45.293391 (+ 92us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:45.298364 (+ 4973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:45.299358 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:45.299365 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:45.299372 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:45.299649 (+ 277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:45.300087 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:45.300093 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:45.300096 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:45.301013 (+ 917us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:45.301490 (+ 477us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:45.301946 (+ 456us) server_negotiation.cc:299] Negotiation successful
0807 11:23:45.302546 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":1259,"thread_start_us":796,"threads_started":1}
I20260807 11:23:45.314528 24480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:45.290638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58638 (local address 127.16.197.62:33957)
0807 11:23:45.291664 (+ 1026us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:45.291673 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:45.293045 (+ 1372us) server_negotiation.cc:407] Connection header received
0807 11:23:45.293256 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:45.293261 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:45.293375 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:45.293526 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:45.295586 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.297102 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.299529 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.300287 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.300438 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:45.305949 (+ 5511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:45.305954 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:45.305957 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:45.305994 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:45.310555 (+ 4561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:45.311643 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:45.311647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:45.311649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:45.311880 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:45.312329 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:45.312333 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:45.312335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:45.312728 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:45.312896 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:45.313519 (+ 623us) server_negotiation.cc:299] Negotiation successful
0807 11:23:45.313753 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":881,"thread_start_us":738,"threads_started":1}
I20260807 11:23:45.314579 24481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:45.290797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.0.0.1:58638)
0807 11:23:45.292878 (+ 2081us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:45.292932 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:45.293113 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:45.293535 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:45.293540 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:45.294712 (+ 1172us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:45.295398 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:45.295423 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.297360 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.297369 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:45.299372 (+ 2003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:45.299385 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.300476 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.300480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:45.300715 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:45.301556 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:45.301583 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:45.305739 (+ 4156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:45.310704 (+ 4965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:45.310716 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:45.310723 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:45.311496 (+ 773us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:45.312005 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:45.312010 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:45.312014 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:45.312196 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:45.312866 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:45.312874 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:45.313124 (+ 250us) client_negotiation.cc:769] Sending connection context
0807 11:23:45.313401 (+ 277us) client_negotiation.cc:240] Negotiation successful
0807 11:23:45.313753 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1735,"mutex_wait_us":85,"thread_start_us":1448,"threads_started":1}
I20260807 11:23:45.316488 24476 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } has no permanent_uuid. Determining permanent_uuid...
I20260807 11:23:45.324211 24307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3f333150234643279ab25bac55bc7419" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: NON_VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: true } } }
I20260807 11:23:45.324749 24307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [sys.catalog]: This master's current role is: LEADER
Aug 07 11:23:45 dist-test-slave-2w3w krb5kdc[23658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101825, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.60@KRBTEST.COM for oryx/127.16.197.61@KRBTEST.COM
I20260807 11:23:45.328780 24482 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419: attempting to promote NON_VOTER b2544d96e5a14f06b63cc24304065247 to VOTER
I20260807 11:23:45.329684 24482 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } }
I20260807 11:23:45.332023 24362 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [term 1 LEARNER]: Refusing update from remote peer 3f333150234643279ab25bac55bc7419: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 11:23:45.332602 24307 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 11:23:45.336642 24481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:45.317573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:41937 (local address 127.0.0.1:47508)
0807 11:23:45.317864 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:45.317892 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:45.317988 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:45.318354 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:45.318358 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:45.318918 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:45.319235 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:45.319244 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.321447 (+ 2203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.321451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:45.322564 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:45.322577 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.323842 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.323849 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:45.324129 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:45.324975 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:45.325011 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:45.329002 (+ 3991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:45.333463 (+ 4461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:45.333468 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:45.333470 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:45.334172 (+ 702us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:45.334559 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:45.334564 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:45.334567 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:45.334849 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:45.335538 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:45.335543 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:45.335718 (+ 175us) client_negotiation.cc:769] Sending connection context
0807 11:23:45.335871 (+ 153us) client_negotiation.cc:240] Negotiation successful
0807 11:23:45.336242 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 11:23:45.336684 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:45.317592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47508 (local address 127.16.197.61:41937)
0807 11:23:45.317881 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:45.317889 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:45.317978 (+ 89us) server_negotiation.cc:407] Connection header received
0807 11:23:45.318091 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:45.318107 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:45.318215 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:45.318331 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:45.319446 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.321273 (+ 1827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.322727 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.323592 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.323740 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:45.329225 (+ 5485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:45.329230 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:45.329233 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:45.329267 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:45.333306 (+ 4039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:45.334306 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:45.334310 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:45.334312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:45.334458 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:45.335015 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:45.335023 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:45.335028 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:45.335403 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:45.335577 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:45.335970 (+ 393us) server_negotiation.cc:299] Negotiation successful
0807 11:23:45.336246 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260807 11:23:45.337716 24482 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, b2544d96e5a14f06b63cc24304065247 (127.16.197.61) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } }
I20260807 11:23:45.338187 24476 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.60" port: 32811 } has no permanent_uuid. Determining permanent_uuid...
I20260807 11:23:45.338647 24362 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, b2544d96e5a14f06b63cc24304065247 (127.16.197.61) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } }
I20260807 11:23:45.342234 24482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3f333150234643279ab25bac55bc7419" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } }
I20260807 11:23:45.342551 24482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:45.342877 24482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3f333150234643279ab25bac55bc7419" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } }
I20260807 11:23:45.343210 24482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:45.344455 24390 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3f333150234643279ab25bac55bc7419" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } }
I20260807 11:23:45.344728 24390 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [sys.catalog]: This master's current role is: FOLLOWER
Aug 07 11:23:45 dist-test-slave-2w3w krb5kdc[23658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101825, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.60@KRBTEST.COM for oryx/127.16.197.60@KRBTEST.COM
I20260807 11:23:45.353392 24481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:45.338959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:32811 (local address 127.0.0.1:46088)
0807 11:23:45.339185 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:45.339199 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:45.339324 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:45.339630 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:45.339634 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:45.340115 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:45.340281 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:45.340289 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.341351 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.341358 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:45.342568 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:45.342583 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.343642 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.343648 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:45.343936 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:45.345139 (+ 1203us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:45.345157 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:45.347823 (+ 2666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:45.351073 (+ 3250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:45.351077 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:45.351080 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:45.351526 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:45.351897 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:45.351901 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:45.351903 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:45.352090 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:45.352576 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:45.352579 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:45.352690 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 11:23:45.352822 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 11:23:45.353028 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":22}
I20260807 11:23:45.353725 24479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:45.339022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46088 (local address 127.16.197.60:32811)
0807 11:23:45.339192 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:45.339198 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:45.339270 (+ 72us) server_negotiation.cc:407] Connection header received
0807 11:23:45.339444 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:45.339449 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:45.339517 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:45.339614 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:45.340416 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.341225 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.342757 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.343468 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.343630 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:45.348025 (+ 4395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:45.348031 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:45.348038 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:45.348071 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:45.350864 (+ 2793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:45.351634 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:45.351638 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:45.351640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:45.351796 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:45.352186 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:45.352189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:45.352191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:45.352477 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:45.352590 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:45.352929 (+ 339us) server_negotiation.cc:299] Negotiation successful
0807 11:23:45.353168 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":37}
I20260807 11:23:45.359349 24476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711: Bootstrap starting.
I20260807 11:23:45.363122 24476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:45.364403 24476 log.cc:824] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:45.367452 24476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711: No bootstrap required, opened a new log
I20260807 11:23:45.376268 24476 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 32811 } }
I20260807 11:23:45.376770 24476 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:45.377015 24476 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2d3da4008c443a6a7596fdac5b97711, State: Initialized, Role: FOLLOWER
I20260807 11:23:45.377733 24476 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 32811 } }
I20260807 11:23:45.378943 24487 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 32811 } } }
I20260807 11:23:45.379452 24487 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 11:23:45.380023 24476 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 11:23:45.391359 24498 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 11:23:45.391541 24498 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 11:23:45.396463 24309 master_runner.cc:428] Detected that this master d2d3da4008c443a6a7596fdac5b97711 is joining an existing cluster
I20260807 11:23:45.396624 24309 master_runner.cc:432] Initiating AddMaster RPC to add 127.16.197.60:32811
I20260807 11:23:45.414217 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 23771
I20260807 11:23:45.431800 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:45.402403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47512 (local address 127.16.197.61:41937)
0807 11:23:45.402635 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:45.402642 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:45.403934 (+ 1292us) server_negotiation.cc:407] Connection header received
0807 11:23:45.404233 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:45.404239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:45.404324 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:45.404420 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:45.406907 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.408560 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.417188 (+ 8628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.418141 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.418259 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:45.422012 (+ 3753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:45.422020 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:45.422026 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:45.422072 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:45.426256 (+ 4184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:45.428708 (+ 2452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:45.428716 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:45.428719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:45.428943 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:45.429480 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:45.429485 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:45.429489 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:45.429902 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:45.430045 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:45.430820 (+ 775us) server_negotiation.cc:299] Negotiation successful
0807 11:23:45.431187 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
Aug 07 11:23:45 dist-test-slave-2w3w krb5kdc[23658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.197.1@KRBTEST.COM
I20260807 11:23:45.433761 24504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:45.402432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:41937 (local address 127.0.0.1:47512)
0807 11:23:45.403732 (+ 1300us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:45.403819 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:45.404060 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:45.404905 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:45.404913 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:45.406121 (+ 1208us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:45.406677 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:45.406695 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.409100 (+ 2405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.409109 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:45.417008 (+ 7899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:45.417023 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.418669 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.418675 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:45.418928 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:45.419905 (+ 977us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:45.419939 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:45.421760 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:45.427488 (+ 5728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:45.427493 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:45.427499 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:45.428526 (+ 1027us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:45.429105 (+ 579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:45.429110 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:45.429114 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:45.429332 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:45.430130 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:45.430135 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:45.430430 (+ 295us) client_negotiation.cc:769] Sending connection context
0807 11:23:45.432667 (+ 2237us) client_negotiation.cc:240] Negotiation successful
0807 11:23:45.433061 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1123,"thread_start_us":896,"threads_started":1}
I20260807 11:23:45.441298 24505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:45.402631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.0.0.1:58646)
0807 11:23:45.410092 (+ 7461us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:45.410117 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:45.410555 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:45.411253 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:45.411259 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:45.412332 (+ 1073us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:45.412985 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:45.413001 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.414538 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.414545 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:45.418338 (+ 3793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:45.418355 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.419954 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.419960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:45.420188 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:45.420942 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:45.420968 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:45.422692 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:45.435480 (+ 12788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:45.435488 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:45.435494 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:45.436165 (+ 671us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:45.437908 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:45.437913 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:45.437916 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:45.438208 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:45.439327 (+ 1119us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:45.439334 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:45.439775 (+ 441us) client_negotiation.cc:769] Sending connection context
0807 11:23:45.439980 (+ 205us) client_negotiation.cc:240] Negotiation successful
0807 11:23:45.440442 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7285,"thread_start_us":7604,"threads_started":1}
I20260807 11:23:45.442515 24480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:45.402026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58646 (local address 127.16.197.62:33957)
0807 11:23:45.402236 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:45.402244 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:45.410223 (+ 7979us) server_negotiation.cc:407] Connection header received
0807 11:23:45.410734 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:45.410742 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:45.410907 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:45.411016 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:45.413183 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.414326 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.418529 (+ 4203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.419444 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.419556 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:45.423464 (+ 3908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:45.423472 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:45.423479 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:45.423526 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:45.435234 (+ 11708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:45.436355 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:45.436363 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:45.436366 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:45.436596 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:45.438708 (+ 2112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:45.438713 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:45.438717 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:45.439177 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:45.439331 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:45.441732 (+ 2401us) server_negotiation.cc:299] Negotiation successful
0807 11:23:45.442069 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 11:23:45.448800 24507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:45.415522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51092 (local address 127.16.197.1:38243)
0807 11:23:45.416461 (+ 939us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:45.416471 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:45.416501 (+ 30us) server_negotiation.cc:407] Connection header received
0807 11:23:45.416592 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:45.416602 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:45.416902 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:45.419226 (+ 2324us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:45.419613 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.421882 (+ 2269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.423895 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.425384 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.426271 (+ 887us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:45.438033 (+ 11762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:45.438058 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:45.438097 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:45.438147 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:45.443737 (+ 5590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:45.444776 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:45.444782 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:45.444785 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:45.445092 (+ 307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:45.445868 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:45.445873 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:45.445876 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:45.446765 (+ 889us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:45.447377 (+ 612us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:45.447639 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 11:23:45.448004 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":729,"thread_start_us":479,"threads_started":1}
I20260807 11:23:45.451001 23904 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:45.451642 24479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:45.404277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46102 (local address 127.16.197.60:32811)
0807 11:23:45.404743 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:45.404764 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:45.416611 (+ 11847us) server_negotiation.cc:407] Connection header received
0807 11:23:45.421421 (+ 4810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:45.421426 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:45.421497 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:45.421570 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:45.427460 (+ 5890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.428645 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.435837 (+ 7192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.437027 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.437129 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:45.442242 (+ 5113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:45.442260 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:45.442264 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:45.442331 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:45.447650 (+ 5319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:45.448668 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:45.448673 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:45.448676 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:45.448854 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:45.449319 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:45.449324 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:45.449327 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:45.449778 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:45.449933 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:45.450609 (+ 676us) server_negotiation.cc:299] Negotiation successful
0807 11:23:45.451046 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":169,"server-negotiator.queue_time_us":322}
I20260807 11:23:45.451722 24506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:45.403934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:32811 (local address 127.0.0.1:46102)
0807 11:23:45.416515 (+ 12581us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:45.416534 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:45.420631 (+ 4097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:45.422903 (+ 2272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:45.422909 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:45.426768 (+ 3859us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:45.427321 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:45.427336 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.431162 (+ 3826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.431167 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:45.433557 (+ 2390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:45.433574 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.437644 (+ 4070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.437654 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:45.437896 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:45.439538 (+ 1642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:45.439561 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:45.441930 (+ 2369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:45.447793 (+ 5863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:45.447797 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:45.447800 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:45.448505 (+ 705us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:45.448994 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:45.448998 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:45.449001 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:45.449193 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:45.450002 (+ 809us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:45.450006 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:45.450319 (+ 313us) client_negotiation.cc:769] Sending connection context
0807 11:23:45.450520 (+ 201us) client_negotiation.cc:240] Negotiation successful
0807 11:23:45.450863 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12400,"thread_start_us":10869,"threads_started":1}
Aug 07 11:23:45 dist-test-slave-2w3w krb5kdc[23658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101822, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.62@KRBTEST.COM for oryx/127.16.197.60@KRBTEST.COM
I20260807 11:23:45.475718 24479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:45.449962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46112 (local address 127.16.197.60:32811)
0807 11:23:45.452796 (+ 2834us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:45.452803 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:45.452822 (+ 19us) server_negotiation.cc:407] Connection header received
0807 11:23:45.452923 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:45.452931 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:45.453067 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:45.453165 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:45.455216 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.457102 (+ 1886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.458923 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.459571 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.459750 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:45.464822 (+ 5072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:45.464829 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:45.464834 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:45.464880 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:45.471398 (+ 6518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:45.472794 (+ 1396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:45.472801 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:45.472805 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:45.473027 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:45.473517 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:45.473523 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:45.473526 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:45.474028 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:45.474200 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:45.474852 (+ 652us) server_negotiation.cc:299] Negotiation successful
0807 11:23:45.475247 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2705,"thread_start_us":17485,"threads_started":1}
I20260807 11:23:45.475795 24508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:45.449378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:32811 (local address 127.0.0.1:46112)
0807 11:23:45.452449 (+ 3071us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:45.452470 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:45.452597 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:45.453386 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:45.453390 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:45.454387 (+ 997us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:45.454949 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:45.454987 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.457291 (+ 2304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.457300 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:45.458773 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:45.458787 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.459764 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.459770 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:45.459982 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:45.460804 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:45.460828 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:45.464543 (+ 3715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:45.471628 (+ 7085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:45.471636 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:45.471640 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:45.472602 (+ 962us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:45.473170 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:45.473176 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:45.473180 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:45.473379 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:45.474250 (+ 871us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:45.474256 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:45.474464 (+ 208us) client_negotiation.cc:769] Sending connection context
0807 11:23:45.474654 (+ 190us) client_negotiation.cc:240] Negotiation successful
0807 11:23:45.475100 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2914,"thread_start_us":2526,"threads_started":1}
I20260807 11:23:45.478168 23704 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.16.197.60:32811: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711"
member_type: NON_VOTER
last_known_addr {
host: "127.16.197.60"
port: 32811
}
attrs {
promote: true
}
}
dest_uuid: "3f333150234643279ab25bac55bc7419"
cas_config_opid_index: 6
I20260807 11:23:45.479357 23704 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: NON_VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: true } }
I20260807 11:23:45.482440 24362 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [term 1 FOLLOWER]: Refusing update from remote peer 3f333150234643279ab25bac55bc7419: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 11:23:45.483326 24482 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 11:23:45.486455 24307 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER d2d3da4008c443a6a7596fdac5b97711 (127.16.197.60) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: NON_VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: true } } }
W20260807 11:23:45.487397 24425 tablet.cc:2369] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:23:45.488001 24449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:45.491855 24362 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER d2d3da4008c443a6a7596fdac5b97711 (127.16.197.60) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: NON_VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: true } } }
I20260807 11:23:45.492457 24307 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.16.197.60:32811
I20260807 11:23:45.493023 24449 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [term 1 FOLLOWER]: Refusing update from remote peer 3f333150234643279ab25bac55bc7419: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 11:23:45.493271 24309 master.cc:537] Master@127.16.197.60:32811 shutting down...
I20260807 11:23:45.496526 24390 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3f333150234643279ab25bac55bc7419" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: NON_VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: true } } }
I20260807 11:23:45.497035 24390 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 11:23:45.498307 24483 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: NON_VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:45.501421 24483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3f333150234643279ab25bac55bc7419" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: NON_VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: true } } }
I20260807 11:23:45.501439 24482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3f333150234643279ab25bac55bc7419" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: NON_VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: true } } }
I20260807 11:23:45.501824 24483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:45.501868 24482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:45.507417 24487 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:23:45.507678 24487 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 32811 } }
I20260807 11:23:45.509112 24487 leader_election.cc:288] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f333150234643279ab25bac55bc7419 (127.16.197.62:33957), b2544d96e5a14f06b63cc24304065247 (127.16.197.61:41937)
I20260807 11:23:45.509884 24362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d2d3da4008c443a6a7596fdac5b97711" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2544d96e5a14f06b63cc24304065247" is_pre_election: true
I20260807 11:23:45.511495 23716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d2d3da4008c443a6a7596fdac5b97711" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f333150234643279ab25bac55bc7419" is_pre_election: true
I20260807 11:23:45.512482 24423 leader_election.cc:302] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2d3da4008c443a6a7596fdac5b97711; no voters: 3f333150234643279ab25bac55bc7419, b2544d96e5a14f06b63cc24304065247
I20260807 11:23:45.513209 24487 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 11:23:45.514694 24309 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:45.515107 24309 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:45.515316 24309 tablet_replica.cc:330] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711: stopping tablet replica
W20260807 11:23:45.517891 23688 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 -> Peer d2d3da4008c443a6a7596fdac5b97711 (127.16.197.60:32811): Couldn't send request to peer d2d3da4008c443a6a7596fdac5b97711. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 11:23:45.522831 24309 master.cc:552] Master@127.16.197.60:32811 shutdown complete.
I20260807 11:23:45.522919 24309 master_runner.cc:305] Clearing existing system tablet
I20260807 11:23:45.523392 24309 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:45.523515 24309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:45.526574 24309 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260807 11:23:45.528216 24514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:45.528551 24309 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:45.528828 24309 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 11:23:45.528932 24309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-2/wal
uuid: "d2d3da4008c443a6a7596fdac5b97711"
format_stamp: "Formatted at 2026-08-07 11:23:45 on dist-test-slave-2w3w"
server_key: "736dca78f02c46f10973b199386807a7"
server_key_iv: "4ca46d5a0bd47bbe85209e98e8ee7b14"
server_key_version: "encryptionkey@0"
I20260807 11:23:45.533958 24309 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:45.541633 24309 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 11:23:45.541810 24309 log.cc:1196] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 11:23:45.542303 24309 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711: Deleting consensus metadata
I20260807 11:23:45.542577 24309 master_runner.cc:315] Copying system tablet from 127.16.197.62:33957
I20260807 11:23:45.546252 24309 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711: tablet copy: Beginning tablet copy session from remote peer at address 127.16.197.62:33957
I20260807 11:23:45.565194 24480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:45.547163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58648 (local address 127.16.197.62:33957)
0807 11:23:45.547403 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:45.547409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:45.547962 (+ 553us) server_negotiation.cc:407] Connection header received
0807 11:23:45.548170 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:45.548176 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:45.548275 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:45.548384 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:45.549613 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.551108 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.553249 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.553995 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.554134 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:45.556943 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:45.556950 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:45.556955 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:45.556996 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:45.561582 (+ 4586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:45.562419 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:45.562426 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:45.562429 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:45.562620 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:45.563176 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:45.563181 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:45.563185 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:45.563688 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:45.563892 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:45.564474 (+ 582us) server_negotiation.cc:299] Negotiation successful
0807 11:23:45.564738 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260807 11:23:45.565824 24521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:45.547116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.0.0.1:58648)
0807 11:23:45.547825 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:45.547850 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:45.547998 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:45.548501 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:45.548506 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:45.549033 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:45.549413 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:45.549427 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.551324 (+ 1897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.551331 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:45.553086 (+ 1755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:45.553101 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.554155 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.554162 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:45.554356 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:45.555261 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:45.555293 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:45.556643 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:45.561763 (+ 5120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:45.561769 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:45.561773 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:45.562282 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:45.562747 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:45.562752 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:45.562756 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:45.563053 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:45.563865 (+ 812us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:45.563870 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:45.564135 (+ 265us) client_negotiation.cc:769] Sending connection context
0807 11:23:45.564287 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 11:23:45.564735 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":438,"threads_started":1}
I20260807 11:23:45.566620 23726 tablet_copy_service.cc:140] P 3f333150234643279ab25bac55bc7419: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer d2d3da4008c443a6a7596fdac5b97711 ({username='oryx', principal='oryx/127.16.197.60@KRBTEST.COM'} at 127.0.0.1:58648)
I20260807 11:23:45.566808 23726 tablet_copy_service.cc:161] P 3f333150234643279ab25bac55bc7419: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer d2d3da4008c443a6a7596fdac5b97711 at {username='oryx', principal='oryx/127.16.197.60@KRBTEST.COM'} at 127.0.0.1:58648: session id = d2d3da4008c443a6a7596fdac5b97711-00000000000000000000000000000000
I20260807 11:23:45.571307 23726 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 11:23:45.573388 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 23771
I20260807 11:23:45.573719 24309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 11:23:45.583282 24309 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711: tablet copy: Starting download of 0 data blocks...
I20260807 11:23:45.583623 24309 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711: tablet copy: Starting download of 1 WAL segments...
I20260807 11:23:45.586463 24309 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 11:23:45.592357 23726 tablet_copy_service.cc:342] P 3f333150234643279ab25bac55bc7419: Request end of tablet copy session d2d3da4008c443a6a7596fdac5b97711-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.16.197.60@KRBTEST.COM'} at 127.0.0.1:58648
I20260807 11:23:45.592541 23726 tablet_copy_service.cc:433] P 3f333150234643279ab25bac55bc7419: ending tablet copy session d2d3da4008c443a6a7596fdac5b97711-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer d2d3da4008c443a6a7596fdac5b97711
I20260807 11:23:45.598052 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:38243
--local_ip_for_outbound_sockets=127.16.197.1
--tserver_master_addrs=127.16.197.62:33957
--webserver_port=42759
--webserver_interface=127.16.197.1
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.1.keytab
--principal=oryx/127.16.197.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:44033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.16.197.62:33957,127.16.197.61:41937
--tserver_master_addrs=127.16.197.62:33957,127.16.197.61:41937,127.16.197.60:32811 with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 11:23:45.605769 24309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:45.608729 24522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:45.610504 24523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:45.618882 24526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:45.624543 24309 server_base.cc:1034] running on GCE node
I20260807 11:23:45.624954 24309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:45.627297 24309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:45.628526 24309 hybrid_clock.cc:648] HybridClock initialized: now 1786101825628461 us; error 53 us; skew 500 ppm
I20260807 11:23:45.628887 24309 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:45 dist-test-slave-2w3w krb5kdc[23658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101825, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:45.631724 24309 init.cc:377] Logged in from keytab as oryx/127.16.197.60@KRBTEST.COM (short username oryx)
I20260807 11:23:45.633355 24309 webserver.cc:460] Webserver started at http://127.16.197.60:39591/ using document root <none> and password file <none>
I20260807 11:23:45.633751 24309 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:45.633875 24309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:45.637249 24309 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260807 11:23:45.639397 24533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:45.640089 24309 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260807 11:23:45.640285 24309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-2/wal
uuid: "d2d3da4008c443a6a7596fdac5b97711"
format_stamp: "Formatted at 2026-08-07 11:23:45 on dist-test-slave-2w3w"
server_key: "736dca78f02c46f10973b199386807a7"
server_key_iv: "4ca46d5a0bd47bbe85209e98e8ee7b14"
server_key_version: "encryptionkey@0"
I20260807 11:23:45.640571 24309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:45.685268 24309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:45.689141 24309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:45.738670 24309 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.60:32811
I20260807 11:23:45.738711 24589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.60:32811 every 8 connection(s)
I20260807 11:23:45.739724 24309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/master-2/data/info.pb
I20260807 11:23:45.750571 24590 sys_catalog.cc:263] Verifying existing consensus state
I20260807 11:23:45.755961 24590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711: Bootstrap starting.
I20260807 11:23:45.771003 24590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 11:23:45.771552 24590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711: Bootstrap complete.
I20260807 11:23:45.772688 24590 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: NON_VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: true } }
I20260807 11:23:45.772982 24590 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d2d3da4008c443a6a7596fdac5b97711, State: Initialized, Role: LEARNER
I20260807 11:23:45.773227 24590 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: NON_VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: true } }
I20260807 11:23:45.776916 24593 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: NON_VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: true } } }
I20260807 11:23:45.777415 24593 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [sys.catalog]: This master's current role is: LEARNER
I20260807 11:23:45.777415 24590 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:23:45.786499 24606 catalog_manager.cc:1259] Loaded cluster ID: 9ded344e6a0f4ebf8ffae9aeddeaab78
I20260807 11:23:45.786634 24606 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711: loading cluster ID for follower catalog manager: success
I20260807 11:23:45.789140 24606 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711: acquiring CA information for follower catalog manager: success
I20260807 11:23:45.791406 24606 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 11:23:45.929049 24508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:45.920329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:32811 (local address 127.0.0.1:46118)
0807 11:23:45.920614 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:45.920639 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:45.920770 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:45.921744 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:45.921751 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:45.921791 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:45.922027 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:45.922042 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.924006 (+ 1964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.924013 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:45.926141 (+ 2128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:45.926161 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.928097 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.928105 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:45.928339 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:45.928380 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 11:23:45.928501 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 11:23:45.928688 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152}
I20260807 11:23:45.929497 24607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:45.920599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46118 (local address 127.16.197.60:32811)
0807 11:23:45.921327 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:45.921342 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:45.921365 (+ 23us) server_negotiation.cc:407] Connection header received
0807 11:23:45.921446 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:45.921452 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:45.921546 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:45.921676 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:45.922218 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.923823 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.926375 (+ 2552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:45.927861 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:45.928027 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:45.928300 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:45.928547 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 11:23:45.928724 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":383,"threads_started":1}
I20260807 11:23:45.930982 24563 raft_consensus.cc:1203] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.7] Dedup: 1.7->[]
I20260807 11:23:45.931695 24593 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f333150234643279ab25bac55bc7419. Latest consensus state: current_term: 1 leader_uuid: "3f333150234643279ab25bac55bc7419" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: NON_VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: true } } }
I20260807 11:23:45.931995 24593 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [sys.catalog]: This master's current role is: LEARNER
W20260807 11:23:46.080314 24524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:46.080688 24524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:46.080912 24524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:46.089980 24524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:46.090165 24524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:46.090394 24524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:23:46.103368 24524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:44033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.1.keytab
--principal=oryx/127.16.197.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:38243
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=42759
--tserver_master_addrs=127.16.197.62:33957,127.16.197.61:41937,127.16.197.60:32811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:46.104871 24524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:46.106531 24524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:46.122959 24612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:46.123703 24613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:46.123767 24616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:46.124630 24524 server_base.cc:1034] running on GCE node
I20260807 11:23:46.125306 24524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:46.126353 24524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:46.127609 24524 hybrid_clock.cc:648] HybridClock initialized: now 1786101826127528 us; error 46 us; skew 500 ppm
I20260807 11:23:46.128005 24524 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:46 dist-test-slave-2w3w krb5kdc[23658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101826, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:46.132158 24524 init.cc:377] Logged in from keytab as oryx/127.16.197.1@KRBTEST.COM (short username oryx)
I20260807 11:23:46.136587 24524 webserver.cc:460] Webserver started at http://127.16.197.1:42759/ using document root <none> and password file <none>
I20260807 11:23:46.137603 24524 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:46.137885 24524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:46.145026 24524 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.003s
I20260807 11:23:46.148723 24624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:46.150215 24524 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 11:23:46.150467 24524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "fcdd78cc9b004378bf2531f78acafc78"
format_stamp: "Formatted at 2026-08-07 11:23:42 on dist-test-slave-2w3w"
server_key: "6332941e3b636e60fb5fd190e11b8cb7"
server_key_iv: "4c2f1cf8ee4a9de70fdb328f6b8f4e09"
server_key_version: "encryptionkey@0"
I20260807 11:23:46.151823 24524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:46.180806 24524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:46.185858 24524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:46.186328 24524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:46.187793 24524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:23:46.189669 24524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:46.189831 24524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:46.189963 24524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:46.190056 24524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:46.259334 24524 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:38243
I20260807 11:23:46.259487 24745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:38243 every 8 connection(s)
I20260807 11:23:46.261081 24524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:23:46.270603 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 24524
I20260807 11:23:46.271025 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 23916
Aug 07 11:23:46 dist-test-slave-2w3w krb5kdc[23658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101826, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.60@KRBTEST.COM
Aug 07 11:23:46 dist-test-slave-2w3w krb5kdc[23658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101826, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM
Aug 07 11:23:46 dist-test-slave-2w3w krb5kdc[23658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101826, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.61@KRBTEST.COM
Aug 07 11:23:46 dist-test-slave-2w3w krb5kdc[23658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.197.2@KRBTEST.COM
I20260807 11:23:46.296795 24752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:46.265869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:48315 (local address 127.16.197.62:33957)
0807 11:23:46.266827 (+ 958us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:46.266836 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:46.267721 (+ 885us) server_negotiation.cc:407] Connection header received
0807 11:23:46.270698 (+ 2977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:46.270706 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:46.270802 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:46.270891 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:46.273609 (+ 2718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:46.274952 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:46.279971 (+ 5019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:46.280703 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:46.281212 (+ 509us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:46.287915 (+ 6703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:46.287924 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:46.287928 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:46.287980 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:46.292984 (+ 5004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:46.294057 (+ 1073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:46.294063 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:46.294066 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:46.294260 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:46.294790 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:46.294795 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:46.294798 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:46.295210 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:46.295400 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:46.296050 (+ 650us) server_negotiation.cc:299] Negotiation successful
0807 11:23:46.296350 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":826,"thread_start_us":708,"threads_started":1}
I20260807 11:23:46.297900 24607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:46.265552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:42669 (local address 127.16.197.60:32811)
0807 11:23:46.265734 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:46.265741 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:46.267657 (+ 1916us) server_negotiation.cc:407] Connection header received
0807 11:23:46.270656 (+ 2999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:46.270662 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:46.270755 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:46.270885 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:46.273807 (+ 2922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:46.275016 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:46.278110 (+ 3094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:46.279114 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:46.279250 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:46.286590 (+ 7340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:46.286599 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:46.286603 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:46.286649 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:46.291582 (+ 4933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:46.292978 (+ 1396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:46.292984 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:46.292989 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:46.293189 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:46.293994 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:46.293999 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:46.294002 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:46.294519 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:46.294774 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:46.297072 (+ 2298us) server_negotiation.cc:299] Negotiation successful
0807 11:23:46.297391 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 11:23:46.298381 24750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:46.265215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.16.197.1:48315)
0807 11:23:46.267477 (+ 2262us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:46.267540 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:46.270387 (+ 2847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:46.271200 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:46.271211 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:46.272128 (+ 917us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:46.273428 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:46.273452 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:46.275270 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:46.275276 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:46.279790 (+ 4514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:46.279805 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:46.281582 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:46.281590 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:46.281820 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:46.282525 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:46.282547 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:46.287539 (+ 4992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:46.293161 (+ 5622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:46.293169 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:46.293176 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:46.293901 (+ 725us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:46.294400 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:46.294403 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:46.294407 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:46.294635 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:46.295373 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:46.295378 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:46.295584 (+ 206us) client_negotiation.cc:769] Sending connection context
0807 11:23:46.297005 (+ 1421us) client_negotiation.cc:240] Negotiation successful
0807 11:23:46.297329 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1819,"mutex_wait_us":105,"spinlock_wait_cycles":82944,"thread_start_us":561,"threads_started":1}
I20260807 11:23:46.298664 24753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:46.267377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:41937 (local address 127.16.197.1:41831)
0807 11:23:46.268679 (+ 1302us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:46.268699 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:46.271548 (+ 2849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:46.274145 (+ 2597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:46.274149 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:46.275129 (+ 980us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:46.276041 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:46.276060 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:46.278919 (+ 2859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:46.278926 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:46.280955 (+ 2029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:46.280972 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:46.282265 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:46.282276 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:46.282551 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:46.283376 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:46.283404 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:46.286912 (+ 3508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:46.293194 (+ 6282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:46.293198 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:46.293201 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:46.293929 (+ 728us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:46.294399 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:46.294404 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:46.294407 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:46.294627 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:46.295902 (+ 1275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:46.295909 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:46.296060 (+ 151us) client_negotiation.cc:769] Sending connection context
0807 11:23:46.296207 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 11:23:46.296681 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1153,"mutex_wait_us":187,"thread_start_us":948,"threads_started":1}
I20260807 11:23:46.299751 24751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:46.265256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:32811 (local address 127.16.197.1:42669)
0807 11:23:46.267499 (+ 2243us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:46.267540 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:46.270391 (+ 2851us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:46.271198 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:46.271213 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:46.272294 (+ 1081us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:46.273600 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:46.273621 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:46.275270 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:46.275276 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:46.277929 (+ 2653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:46.277944 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:46.279695 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:46.279704 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:46.280139 (+ 435us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:46.281248 (+ 1109us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:46.281341 (+ 93us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:46.286315 (+ 4974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:46.291805 (+ 5490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:46.291824 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:46.291847 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:46.292793 (+ 946us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:46.293367 (+ 574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:46.293372 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:46.293376 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:46.293835 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:46.294875 (+ 1040us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:46.294884 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:46.295540 (+ 656us) client_negotiation.cc:769] Sending connection context
0807 11:23:46.295912 (+ 372us) client_negotiation.cc:240] Negotiation successful
0807 11:23:46.296549 (+ 637us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2080,"mutex_wait_us":558,"spinlock_wait_cycles":4864,"thread_start_us":1119,"threads_started":1}
I20260807 11:23:46.300489 24747 heartbeater.cc:347] Connected to a master server at 127.16.197.62:33957
I20260807 11:23:46.300640 24755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:46.267808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:41831 (local address 127.16.197.61:41937)
0807 11:23:46.268465 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:46.268473 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:46.268797 (+ 324us) server_negotiation.cc:407] Connection header received
0807 11:23:46.273779 (+ 4982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:46.273795 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:46.273939 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:46.277362 (+ 3423us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:46.277554 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:46.278712 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:46.281229 (+ 2517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:46.281990 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:46.283082 (+ 1092us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:46.287209 (+ 4127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:46.287215 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:46.287220 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:46.287302 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:46.293064 (+ 5762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:46.294091 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:46.294096 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:46.294099 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:46.294254 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:46.294799 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:46.294803 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:46.294807 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:46.295749 (+ 942us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:46.296439 (+ 690us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:46.299957 (+ 3518us) server_negotiation.cc:299] Negotiation successful
0807 11:23:46.300201 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":416,"threads_started":1}
I20260807 11:23:46.300987 24747 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:46.301493 24746 heartbeater.cc:347] Connected to a master server at 127.16.197.60:32811
I20260807 11:23:46.301574 24748 heartbeater.cc:347] Connected to a master server at 127.16.197.61:41937
I20260807 11:23:46.301641 24746 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:46.301752 24748 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:46.302522 24746 heartbeater.cc:511] Master 127.16.197.60:32811 requested a full tablet report, sending...
I20260807 11:23:46.302614 24747 heartbeater.cc:511] Master 127.16.197.62:33957 requested a full tablet report, sending...
I20260807 11:23:46.302778 24748 heartbeater.cc:511] Master 127.16.197.61:41937 requested a full tablet report, sending...
I20260807 11:23:46.305537 24352 ts_manager.cc:194] Registered new tserver with Master: fcdd78cc9b004378bf2531f78acafc78 (127.16.197.1:38243)
I20260807 11:23:46.305526 24553 ts_manager.cc:194] Registered new tserver with Master: fcdd78cc9b004378bf2531f78acafc78 (127.16.197.1:38243)
I20260807 11:23:46.307109 24757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:46.272876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44350 (local address 127.16.197.2:41871)
0807 11:23:46.276939 (+ 4063us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:46.276945 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:46.276968 (+ 23us) server_negotiation.cc:407] Connection header received
0807 11:23:46.277077 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:46.277083 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:46.277343 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:46.277547 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:46.280766 (+ 3219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:46.282727 (+ 1961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:46.285165 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:46.286375 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:46.286515 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:46.294287 (+ 7772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:46.294307 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:46.294326 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:46.294398 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:46.299733 (+ 5335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:46.303465 (+ 3732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:46.303475 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:46.303479 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:46.303776 (+ 297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:46.304302 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:46.304307 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:46.304309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:46.304919 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:46.305334 (+ 415us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:46.305720 (+ 386us) server_negotiation.cc:299] Negotiation successful
0807 11:23:46.306044 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3872,"thread_start_us":3642,"threads_started":1}
I20260807 11:23:46.307350 23704 ts_manager.cc:194] Re-registered known tserver with Master: fcdd78cc9b004378bf2531f78acafc78 (127.16.197.1:38243)
I20260807 11:23:46.308274 23704 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.197.1@KRBTEST.COM'} at 127.16.197.1:48315
I20260807 11:23:46.308465 24047 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:46.373732 24511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3f333150234643279ab25bac55bc7419" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: NON_VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: true } } }
I20260807 11:23:46.374261 24511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:46.376215 24760 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419: attempting to promote NON_VOTER d2d3da4008c443a6a7596fdac5b97711 to VOTER
I20260807 11:23:46.376878 24760 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: false } }
I20260807 11:23:46.379212 24362 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [term 1 FOLLOWER]: Refusing update from remote peer 3f333150234643279ab25bac55bc7419: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 11:23:46.379279 24563 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [term 1 LEARNER]: Refusing update from remote peer 3f333150234643279ab25bac55bc7419: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 11:23:46.379985 24760 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 11:23:46.380482 24759 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 11:23:46.383421 24760 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, d2d3da4008c443a6a7596fdac5b97711 (127.16.197.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: false } } }
I20260807 11:23:46.384164 24362 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, d2d3da4008c443a6a7596fdac5b97711 (127.16.197.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: false } } }
I20260807 11:23:46.389586 24759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3f333150234643279ab25bac55bc7419" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: false } } }
I20260807 11:23:46.389962 24759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:46.390388 24759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3f333150234643279ab25bac55bc7419" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: false } } }
I20260807 11:23:46.390887 24759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:46.397238 24763 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3f333150234643279ab25bac55bc7419" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: false } } }
I20260807 11:23:46.397782 24763 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 11:23:46.397827 24760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3f333150234643279ab25bac55bc7419" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: false } } }
I20260807 11:23:46.398125 24760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:46.398669 24562 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, d2d3da4008c443a6a7596fdac5b97711 (127.16.197.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: false } } }
I20260807 11:23:46.404211 24593 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3f333150234643279ab25bac55bc7419" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: false } } }
I20260807 11:23:46.404544 24593 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 11:23:46.408461 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 23916
I20260807 11:23:46.426465 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:41871
--local_ip_for_outbound_sockets=127.16.197.2
--tserver_master_addrs=127.16.197.62:33957
--webserver_port=38979
--webserver_interface=127.16.197.2
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.2.keytab
--principal=oryx/127.16.197.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:44033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.16.197.62:33957,127.16.197.61:41937
--tserver_master_addrs=127.16.197.62:33957,127.16.197.61:41937,127.16.197.60:32811 with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:46.769191 24765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:46.769531 24765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:46.769697 24765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:46.777184 24765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:46.777357 24765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:46.777578 24765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:23:46.788623 24765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:44033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.2.keytab
--principal=oryx/127.16.197.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:41871
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=38979
--tserver_master_addrs=127.16.197.62:33957,127.16.197.61:41937,127.16.197.60:32811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:46.789992 24765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:46.791419 24765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:46.806311 24771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:46.806313 24774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:46.806375 24772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:46.807819 24765 server_base.cc:1034] running on GCE node
I20260807 11:23:46.808550 24765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:46.809590 24765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:46.810791 24765 hybrid_clock.cc:648] HybridClock initialized: now 1786101826810730 us; error 54 us; skew 500 ppm
I20260807 11:23:46.811189 24765 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:46 dist-test-slave-2w3w krb5kdc[23658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101826, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:46.814805 24765 init.cc:377] Logged in from keytab as oryx/127.16.197.2@KRBTEST.COM (short username oryx)
I20260807 11:23:46.816543 24765 webserver.cc:460] Webserver started at http://127.16.197.2:38979/ using document root <none> and password file <none>
I20260807 11:23:46.817222 24765 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:46.817344 24765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:46.822825 24765 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:23:46.825753 24781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:46.827212 24765 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:23:46.827346 24765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "af5f0cfe2e664d478c15b24cb6071baf"
format_stamp: "Formatted at 2026-08-07 11:23:43 on dist-test-slave-2w3w"
server_key: "9bec3306606d2606a9fef95f88df335e"
server_key_iv: "237e4031a16f23497e70993eeb4167db"
server_key_version: "encryptionkey@0"
I20260807 11:23:46.828358 24765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:46.846297 24765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:46.850811 24765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:46.851192 24765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:46.852181 24765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:23:46.853575 24765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:46.853678 24765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:46.853760 24765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:46.853840 24765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:46.889282 24765 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:41871
I20260807 11:23:46.889387 24901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:41871 every 8 connection(s)
I20260807 11:23:46.890857 24765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:23:46.892069 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 24765
I20260807 11:23:46.892388 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 24059
Aug 07 11:23:46 dist-test-slave-2w3w krb5kdc[23658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.197.3@KRBTEST.COM
Aug 07 11:23:46 dist-test-slave-2w3w krb5kdc[23658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101826, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM
Aug 07 11:23:46 dist-test-slave-2w3w krb5kdc[23658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101826, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.61@KRBTEST.COM
Aug 07 11:23:46 dist-test-slave-2w3w krb5kdc[23658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101826, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.60@KRBTEST.COM
I20260807 11:23:46.932116 24906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:46.893669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42260 (local address 127.16.197.3:39535)
0807 11:23:46.904412 (+ 10743us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:46.904419 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:46.905321 (+ 902us) server_negotiation.cc:407] Connection header received
0807 11:23:46.905650 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:46.905661 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:46.905988 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:46.906163 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:46.908943 (+ 2780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:46.910558 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:46.912584 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:46.913474 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:46.913582 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:46.921710 (+ 8128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:46.921729 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:46.921749 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:46.921808 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:46.927187 (+ 5379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:46.928304 (+ 1117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:46.928316 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:46.928319 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:46.928525 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:46.929106 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:46.929112 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:46.929129 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:46.929690 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:46.930084 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:46.931230 (+ 1146us) server_negotiation.cc:299] Negotiation successful
0807 11:23:46.931568 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10565,"thread_start_us":10414,"threads_started":1}
I20260807 11:23:46.932307 24912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:46.898450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:52947 (local address 127.16.197.60:32811)
0807 11:23:46.901123 (+ 2673us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:46.901133 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:46.906248 (+ 5115us) server_negotiation.cc:407] Connection header received
0807 11:23:46.907050 (+ 802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:46.907055 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:46.907161 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:46.907264 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:46.910516 (+ 3252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:46.911744 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:46.915286 (+ 3542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:46.916024 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:46.916145 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:46.923379 (+ 7234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:46.923388 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:46.923394 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:46.923438 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:46.927812 (+ 4374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:46.928919 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:46.928925 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:46.928929 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:46.929216 (+ 287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:46.929893 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:46.929898 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:46.929901 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:46.930278 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:46.930436 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:46.931544 (+ 1108us) server_negotiation.cc:299] Negotiation successful
0807 11:23:46.931798 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2476,"thread_start_us":2309,"threads_started":1}
I20260807 11:23:46.933696 24910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:46.897906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:32811 (local address 127.16.197.2:52947)
0807 11:23:46.906109 (+ 8203us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:46.906127 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:46.906246 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:46.907570 (+ 1324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:46.907581 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:46.909004 (+ 1423us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:46.910295 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:46.910319 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:46.911972 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:46.911976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:46.915095 (+ 3119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:46.915112 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:46.916506 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:46.916512 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:46.916716 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:46.917601 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:46.917627 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:46.923131 (+ 5504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:46.928020 (+ 4889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:46.928029 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:46.928050 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:46.928772 (+ 722us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:46.929369 (+ 597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:46.929374 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:46.929378 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:46.929701 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:46.930542 (+ 841us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:46.930550 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:46.931053 (+ 503us) client_negotiation.cc:769] Sending connection context
0807 11:23:46.931443 (+ 390us) client_negotiation.cc:240] Negotiation successful
0807 11:23:46.932004 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8057,"mutex_wait_us":377,"spinlock_wait_cycles":25216,"thread_start_us":7860,"threads_started":1}
I20260807 11:23:46.933734 24192 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:46.933736 24909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:46.895099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.16.197.2:46769)
0807 11:23:46.896222 (+ 1123us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:46.896304 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:46.899963 (+ 3659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:46.907571 (+ 7608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:46.907583 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:46.909024 (+ 1441us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:46.910291 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:46.910319 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:46.911937 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:46.911943 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:46.914450 (+ 2507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:46.914465 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:46.916229 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:46.916236 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:46.916432 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:46.917440 (+ 1008us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:46.917484 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:46.921725 (+ 4241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:46.928202 (+ 6477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:46.928209 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:46.928216 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:46.928841 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:46.929779 (+ 938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:46.929784 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:46.929789 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:46.930000 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:46.930678 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:46.930683 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:46.931062 (+ 379us) client_negotiation.cc:769] Sending connection context
0807 11:23:46.932128 (+ 1066us) client_negotiation.cc:240] Negotiation successful
0807 11:23:46.933059 (+ 931us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"mutex_wait_us":320,"spinlock_wait_cycles":90112,"thread_start_us":10117,"threads_started":1}
I20260807 11:23:46.934248 24907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:46.894399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:46769 (local address 127.16.197.62:33957)
0807 11:23:46.907058 (+ 12659us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:46.907064 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:46.907102 (+ 38us) server_negotiation.cc:407] Connection header received
0807 11:23:46.907159 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:46.907164 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:46.907246 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:46.907355 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:46.910521 (+ 3166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:46.911745 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:46.914647 (+ 2902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:46.915420 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:46.915546 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:46.923151 (+ 7605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:46.923158 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:46.923162 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:46.923212 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:46.928068 (+ 4856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:46.929387 (+ 1319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:46.929392 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:46.929395 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:46.929661 (+ 266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:46.930122 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:46.930127 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:46.930129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:46.930567 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:46.930736 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:46.933506 (+ 2770us) server_negotiation.cc:299] Negotiation successful
0807 11:23:46.933789 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12490,"thread_start_us":12378,"threads_started":1}
I20260807 11:23:46.935855 24902 heartbeater.cc:347] Connected to a master server at 127.16.197.60:32811
I20260807 11:23:46.936405 24902 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:46.936962 24908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:46.894745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:53819 (local address 127.16.197.61:41937)
0807 11:23:46.906698 (+ 11953us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:46.906707 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:46.906725 (+ 18us) server_negotiation.cc:407] Connection header received
0807 11:23:46.906790 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:46.906795 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:46.906887 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:46.907023 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:46.910512 (+ 3489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:46.911785 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:46.915194 (+ 3409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:46.915833 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:46.915977 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:46.922773 (+ 6796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:46.922781 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:46.922787 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:46.922837 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:46.927979 (+ 5142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:46.929702 (+ 1723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:46.929709 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:46.929711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:46.930316 (+ 605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:46.931377 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:46.931382 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:46.931385 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:46.931786 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:46.931925 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:46.936215 (+ 4290us) server_negotiation.cc:299] Negotiation successful
0807 11:23:46.936487 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11814,"thread_start_us":11699,"threads_started":1}
I20260807 11:23:46.937106 24903 heartbeater.cc:347] Connected to a master server at 127.16.197.62:33957
I20260807 11:23:46.937968 24903 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:46.938575 24902 heartbeater.cc:511] Master 127.16.197.60:32811 requested a full tablet report, sending...
I20260807 11:23:46.938671 24903 heartbeater.cc:511] Master 127.16.197.62:33957 requested a full tablet report, sending...
I20260807 11:23:46.939827 24914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:46.895163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:41937 (local address 127.16.197.2:53819)
0807 11:23:46.905648 (+ 10485us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:46.905674 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:46.905821 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:46.907738 (+ 1917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:46.907742 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:46.909030 (+ 1288us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:46.910292 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:46.910318 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:46.911970 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:46.911976 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:46.914925 (+ 2949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:46.914938 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:46.916092 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:46.916100 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:46.916437 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:46.917440 (+ 1003us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:46.917484 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:46.922498 (+ 5014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:46.928964 (+ 6466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:46.928969 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:46.928972 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:46.929553 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:46.930468 (+ 915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:46.930473 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:46.930476 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:46.931215 (+ 739us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:46.934925 (+ 3710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:46.934933 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:46.935805 (+ 872us) client_negotiation.cc:769] Sending connection context
0807 11:23:46.936070 (+ 265us) client_negotiation.cc:240] Negotiation successful
0807 11:23:46.936385 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10345,"mutex_wait_us":256,"spinlock_wait_cycles":80384,"thread_start_us":1343,"threads_started":1}
I20260807 11:23:46.940814 23704 ts_manager.cc:194] Re-registered known tserver with Master: af5f0cfe2e664d478c15b24cb6071baf (127.16.197.2:41871)
I20260807 11:23:46.941169 24553 ts_manager.cc:194] Registered new tserver with Master: af5f0cfe2e664d478c15b24cb6071baf (127.16.197.2:41871)
I20260807 11:23:46.941533 24904 heartbeater.cc:347] Connected to a master server at 127.16.197.61:41937
I20260807 11:23:46.941695 24904 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:46.942286 23704 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.197.2@KRBTEST.COM'} at 127.16.197.2:46769
I20260807 11:23:46.942365 24904 heartbeater.cc:511] Master 127.16.197.61:41937 requested a full tablet report, sending...
I20260807 11:23:46.943822 24352 ts_manager.cc:194] Registered new tserver with Master: af5f0cfe2e664d478c15b24cb6071baf (127.16.197.2:41871)
I20260807 11:23:47.025991 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 24059
I20260807 11:23:47.044404 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:39535
--local_ip_for_outbound_sockets=127.16.197.3
--tserver_master_addrs=127.16.197.62:33957
--webserver_port=33565
--webserver_interface=127.16.197.3
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.3.keytab
--principal=oryx/127.16.197.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:44033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.16.197.62:33957,127.16.197.61:41937
--tserver_master_addrs=127.16.197.62:33957,127.16.197.61:41937,127.16.197.60:32811 with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 11:23:47.311367 24747 heartbeater.cc:503] Master 127.16.197.62:33957 was elected leader, sending a full tablet report...
W20260807 11:23:47.394084 24916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:47.394387 24916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:47.394484 24916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:47.401912 24916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:47.402026 24916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:47.402141 24916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:23:47.412356 24916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:44033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.3.keytab
--principal=oryx/127.16.197.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:39535
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=33565
--tserver_master_addrs=127.16.197.62:33957,127.16.197.61:41937,127.16.197.60:32811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:47.413692 24916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:47.415009 24916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:47.428782 24924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:47.428891 24926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:47.428887 24923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:47.429780 24916 server_base.cc:1034] running on GCE node
I20260807 11:23:47.430454 24916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:47.431566 24916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:47.432816 24916 hybrid_clock.cc:648] HybridClock initialized: now 1786101827432745 us; error 49 us; skew 500 ppm
I20260807 11:23:47.433260 24916 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:47 dist-test-slave-2w3w krb5kdc[23658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101827, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:47.436578 24916 init.cc:377] Logged in from keytab as oryx/127.16.197.3@KRBTEST.COM (short username oryx)
I20260807 11:23:47.438190 24916 webserver.cc:460] Webserver started at http://127.16.197.3:33565/ using document root <none> and password file <none>
I20260807 11:23:47.438851 24916 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:47.439023 24916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:47.444582 24916 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260807 11:23:47.447441 24933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:47.448756 24916 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 11:23:47.448912 24916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "2fd50e45dcc24d218e3c7f4951e5bf86"
format_stamp: "Formatted at 2026-08-07 11:23:43 on dist-test-slave-2w3w"
server_key: "cf11bcc2b12a78399ed70de655d629cc"
server_key_iv: "c400b3e2e74ad8842f2bf72602a52496"
server_key_version: "encryptionkey@0"
I20260807 11:23:47.449846 24916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:47.466771 24916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:47.470738 24916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:47.471086 24916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:47.472118 24916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:23:47.473521 24916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:47.473637 24916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:47.473779 24916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:47.473896 24916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:47.510288 24916 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:39535
I20260807 11:23:47.510385 25054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:39535 every 8 connection(s)
I20260807 11:23:47.511905 24916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:23:47.513008 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 24916
Aug 07 11:23:47 dist-test-slave-2w3w krb5kdc[23658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101827, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.60@KRBTEST.COM
Aug 07 11:23:47 dist-test-slave-2w3w krb5kdc[23658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101827, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM
I20260807 11:23:47.548399 25058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:47.515199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:32811 (local address 127.16.197.3:42051)
0807 11:23:47.516147 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:47.516197 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:47.520188 (+ 3991us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:47.521654 (+ 1466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:47.521672 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:47.522737 (+ 1065us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:47.523824 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:47.523854 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.525944 (+ 2090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.525959 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:47.528900 (+ 2941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:47.528930 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.530712 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.530722 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:47.531166 (+ 444us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:47.532312 (+ 1146us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:47.532355 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:47.536763 (+ 4408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:47.542225 (+ 5462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:47.542239 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:47.542259 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:47.543038 (+ 779us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:47.543534 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:47.543540 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:47.543546 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:47.543848 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:47.544680 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:47.544690 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:47.545547 (+ 857us) client_negotiation.cc:769] Sending connection context
0807 11:23:47.545948 (+ 401us) client_negotiation.cc:240] Negotiation successful
0807 11:23:47.546555 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":608,"threads_started":1}
I20260807 11:23:47.548668 25059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:47.515742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:42051 (local address 127.16.197.60:32811)
0807 11:23:47.521044 (+ 5302us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:47.521049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:47.521066 (+ 17us) server_negotiation.cc:407] Connection header received
0807 11:23:47.521119 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:47.521123 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:47.521186 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:47.521316 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:47.524113 (+ 2797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.525668 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.529191 (+ 3523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.530124 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.530276 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:47.537140 (+ 6864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:47.537151 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:47.537159 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:47.537220 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:47.541982 (+ 4762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:47.543199 (+ 1217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:47.543207 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:47.543212 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:47.543378 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:47.544000 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:47.544004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:47.544008 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:47.544378 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:47.544547 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:47.546993 (+ 2446us) server_negotiation.cc:299] Negotiation successful
0807 11:23:47.547283 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5191,"thread_start_us":5044,"threads_started":1}
I20260807 11:23:47.552186 25055 heartbeater.cc:347] Connected to a master server at 127.16.197.60:32811
I20260807 11:23:47.552479 25055 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:47.553696 25055 heartbeater.cc:511] Master 127.16.197.60:32811 requested a full tablet report, sending...
I20260807 11:23:47.556613 24553 ts_manager.cc:194] Registered new tserver with Master: 2fd50e45dcc24d218e3c7f4951e5bf86 (127.16.197.3:39535)
Aug 07 11:23:47 dist-test-slave-2w3w krb5kdc[23658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101827, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.61@KRBTEST.COM
I20260807 11:23:47.560480 25060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:47.519793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:53793 (local address 127.16.197.62:33957)
0807 11:23:47.538029 (+ 18236us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:47.538040 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:47.538068 (+ 28us) server_negotiation.cc:407] Connection header received
0807 11:23:47.538147 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:47.538156 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:47.538262 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:47.538420 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:47.539723 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.541243 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.543234 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.544121 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.544279 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:47.549268 (+ 4989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:47.549276 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:47.549281 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:47.549324 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:47.554540 (+ 5216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:47.555790 (+ 1250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:47.555796 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:47.555800 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:47.556010 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:47.558357 (+ 2347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:47.558365 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:47.558371 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:47.558796 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:47.559010 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:47.559576 (+ 566us) server_negotiation.cc:299] Negotiation successful
0807 11:23:47.559835 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18045,"thread_start_us":18763,"threads_started":1}
I20260807 11:23:47.560523 25061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:47.519809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.16.197.3:53793)
0807 11:23:47.523442 (+ 3633us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:47.523469 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:47.523610 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:47.538479 (+ 14869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:47.538486 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:47.539209 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:47.539554 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:47.539571 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.541410 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.541416 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:47.543097 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:47.543111 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.544308 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.544314 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:47.544509 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:47.545185 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:47.545206 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:47.548316 (+ 3110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:47.554762 (+ 6446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:47.554766 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:47.554768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:47.555608 (+ 840us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:47.556155 (+ 547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:47.556163 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:47.556169 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:47.556456 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:47.559000 (+ 2544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:47.559009 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:47.559231 (+ 222us) client_negotiation.cc:769] Sending connection context
0807 11:23:47.559429 (+ 198us) client_negotiation.cc:240] Negotiation successful
0807 11:23:47.559922 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3496,"thread_start_us":3301,"threads_started":1}
I20260807 11:23:47.562047 25056 heartbeater.cc:347] Connected to a master server at 127.16.197.62:33957
I20260807 11:23:47.562282 25056 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:47.562983 25056 heartbeater.cc:511] Master 127.16.197.62:33957 requested a full tablet report, sending...
I20260807 11:23:47.564291 23704 ts_manager.cc:194] Re-registered known tserver with Master: 2fd50e45dcc24d218e3c7f4951e5bf86 (127.16.197.3:39535)
I20260807 11:23:47.565745 23704 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.197.3@KRBTEST.COM'} at 127.16.197.3:53793
I20260807 11:23:47.570381 25064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:47.524423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:41937 (local address 127.16.197.3:46233)
0807 11:23:47.546031 (+ 21608us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:47.546058 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:47.546237 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:47.546901 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:47.546906 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:47.547674 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:47.548463 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:47.548479 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.550940 (+ 2461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.550946 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:47.553003 (+ 2057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:47.553017 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.554786 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.554796 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:47.555104 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:47.556297 (+ 1193us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:47.556323 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:47.560853 (+ 4530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:47.566367 (+ 5514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:47.566373 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:47.566377 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:47.567115 (+ 738us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:47.567786 (+ 671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:47.567791 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:47.567795 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:47.568055 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:47.568858 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:47.568863 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:47.569092 (+ 229us) client_negotiation.cc:769] Sending connection context
0807 11:23:47.569227 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 11:23:47.569689 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21418,"thread_start_us":25938,"threads_started":1}
I20260807 11:23:47.571828 25065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:47.524862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:46233 (local address 127.16.197.61:41937)
0807 11:23:47.545311 (+ 20449us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:47.545320 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:47.546383 (+ 1063us) server_negotiation.cc:407] Connection header received
0807 11:23:47.546472 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:47.546479 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:47.546582 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:47.546701 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:47.548665 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.550200 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.553184 (+ 2984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.554082 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.554191 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:47.561155 (+ 6964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:47.561162 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:47.561173 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:47.561210 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:47.566180 (+ 4970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:47.567344 (+ 1164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:47.567349 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:47.567352 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:47.567554 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:47.568228 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:47.568233 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:47.568238 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:47.568687 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:47.570790 (+ 2103us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:47.570944 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 11:23:47.571246 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20258,"thread_start_us":20468,"threads_started":1}
I20260807 11:23:47.572911 25062 heartbeater.cc:347] Connected to a master server at 127.16.197.61:41937
I20260807 11:23:47.573052 25062 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:47.573498 25062 heartbeater.cc:511] Master 127.16.197.61:41937 requested a full tablet report, sending...
I20260807 11:23:47.575692 24352 ts_manager.cc:194] Registered new tserver with Master: 2fd50e45dcc24d218e3c7f4951e5bf86 (127.16.197.3:39535)
I20260807 11:23:47.595593 25070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:47.563228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47516 (local address 127.16.197.61:41937)
0807 11:23:47.571218 (+ 7990us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:47.571224 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:47.572441 (+ 1217us) server_negotiation.cc:407] Connection header received
0807 11:23:47.572522 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:47.572530 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:47.572630 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:47.573947 (+ 1317us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:47.576630 (+ 2683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.577915 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.580024 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.581043 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.581180 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:47.586055 (+ 4875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:47.586066 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:47.586074 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:47.586120 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:47.591234 (+ 5114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:47.592364 (+ 1130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:47.592371 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:47.592374 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:47.592579 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:47.593129 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:47.593136 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:47.593142 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:47.593782 (+ 640us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:47.594159 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:47.594453 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 11:23:47.594765 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7849,"thread_start_us":7661,"threads_started":1}
I20260807 11:23:47.597995 25060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:47.562053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58658 (local address 127.16.197.62:33957)
0807 11:23:47.562574 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:47.562595 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:47.575507 (+ 12912us) server_negotiation.cc:407] Connection header received
0807 11:23:47.575976 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:47.575982 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:47.576071 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:47.576145 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:47.580604 (+ 4459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.581755 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.584028 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.584554 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.584707 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:47.588429 (+ 3722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:47.588434 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:47.588438 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:47.588474 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:47.592528 (+ 4054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:47.593517 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:47.593523 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:47.593526 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:47.595202 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:47.596023 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:47.596029 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:47.596033 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:47.596526 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:47.596851 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:47.597489 (+ 638us) server_negotiation.cc:299] Negotiation successful
0807 11:23:47.597710 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295}
I20260807 11:23:47.600169 25059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:47.570754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46128 (local address 127.16.197.60:32811)
0807 11:23:47.570999 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:47.571006 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:47.575550 (+ 4544us) server_negotiation.cc:407] Connection header received
0807 11:23:47.575975 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:47.575982 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:47.576071 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:47.576146 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:47.580796 (+ 4650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.582025 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.584849 (+ 2824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.585707 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.585823 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:47.590945 (+ 5122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:47.590984 (+ 39us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:47.590990 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:47.591044 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:47.595622 (+ 4578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:47.596543 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:47.596549 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:47.596553 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:47.596777 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:47.597299 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:47.597305 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:47.597308 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:47.597790 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:47.598188 (+ 398us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:47.599399 (+ 1211us) server_negotiation.cc:299] Negotiation successful
0807 11:23:47.599707 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 11:23:47.603526 23704 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58658:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 11:23:47.613168 23704 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:23:47.664822 25086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:47.642131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44356 (local address 127.16.197.2:41871)
0807 11:23:47.650072 (+ 7941us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:47.650086 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:47.650130 (+ 44us) server_negotiation.cc:407] Connection header received
0807 11:23:47.650809 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:47.650824 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:47.651274 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:47.652764 (+ 1490us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:47.653041 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.655084 (+ 2043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.661615 (+ 6531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.663562 (+ 1947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.663727 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:47.663954 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:47.664244 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 11:23:47.664435 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7710,"thread_start_us":7525,"threads_started":1}
I20260807 11:23:47.665087 25087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:47.641336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:41871 (local address 127.0.0.1:44356)
0807 11:23:47.644454 (+ 3118us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:47.644478 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:47.644627 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:47.651878 (+ 7251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:47.651887 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:47.651931 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:47.652494 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:47.652511 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.659101 (+ 6590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.659108 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:47.661326 (+ 2218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:47.661346 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.663808 (+ 2462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.663815 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:47.664023 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:47.664049 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 11:23:47.664158 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 11:23:47.664304 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2954,"mutex_wait_us":26,"thread_start_us":1815,"threads_started":1}
I20260807 11:23:47.669879 24833 tablet_service.cc:1467] Processing CreateTablet for tablet c86a0a8977b540db87048f7fd77a8e8b (DEFAULT_TABLE table=test-table [id=e67b1a84945a406887249a3edf6cf051]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:47.671772 24833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c86a0a8977b540db87048f7fd77a8e8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:47.673359 25084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:47.640960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51108 (local address 127.16.197.1:38243)
0807 11:23:47.656551 (+ 15591us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:47.656560 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:47.656592 (+ 32us) server_negotiation.cc:407] Connection header received
0807 11:23:47.656699 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:47.656704 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:47.658662 (+ 1958us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:47.658888 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:47.659657 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.661001 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.665464 (+ 4463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.671919 (+ 6455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.672097 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:47.672335 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:47.672817 (+ 482us) server_negotiation.cc:299] Negotiation successful
0807 11:23:47.673017 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15407,"thread_start_us":15257,"threads_started":1}
I20260807 11:23:47.673821 25085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:47.640651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:38243 (local address 127.0.0.1:51108)
0807 11:23:47.642726 (+ 2075us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:47.642754 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:47.643000 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:47.658854 (+ 15854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:47.658858 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:47.658879 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:47.659445 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:47.659458 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.661167 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.661175 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:47.665229 (+ 4054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:47.665253 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.672239 (+ 6986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.672248 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:47.672549 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:47.672589 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 11:23:47.672692 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 11:23:47.672940 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1833,"thread_start_us":16320,"threads_started":1}
I20260807 11:23:47.680439 25083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:47.643603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:39535 (local address 127.0.0.1:42274)
0807 11:23:47.657223 (+ 13620us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:47.657239 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:47.657338 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:47.658280 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:47.658284 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:47.658307 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:47.658605 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:47.658613 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.660514 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.660520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:47.663292 (+ 2772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:47.663308 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.679256 (+ 15948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.679263 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:47.679498 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:47.679530 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 11:23:47.679655 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 11:23:47.679829 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13512,"mutex_wait_us":2,"thread_start_us":846,"threads_started":1}
I20260807 11:23:47.678857 24676 tablet_service.cc:1467] Processing CreateTablet for tablet c86a0a8977b540db87048f7fd77a8e8b (DEFAULT_TABLE table=test-table [id=e67b1a84945a406887249a3edf6cf051]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:47.681206 25088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:47.644260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42274 (local address 127.16.197.3:39535)
0807 11:23:47.657781 (+ 13521us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:47.657787 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:47.657815 (+ 28us) server_negotiation.cc:407] Connection header received
0807 11:23:47.657885 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:47.657889 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:47.658072 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:47.658294 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:47.658829 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.660352 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.667386 (+ 7034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.669425 (+ 2039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.669575 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:47.669819 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:47.679782 (+ 9963us) server_negotiation.cc:299] Negotiation successful
0807 11:23:47.680032 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13343,"thread_start_us":13148,"threads_started":1}
I20260807 11:23:47.681926 24676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c86a0a8977b540db87048f7fd77a8e8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:47.686460 24985 tablet_service.cc:1467] Processing CreateTablet for tablet c86a0a8977b540db87048f7fd77a8e8b (DEFAULT_TABLE table=test-table [id=e67b1a84945a406887249a3edf6cf051]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:47.688521 24985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c86a0a8977b540db87048f7fd77a8e8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:47.692392 25089 tablet_bootstrap.cc:492] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf: Bootstrap starting.
I20260807 11:23:47.698618 25089 tablet_bootstrap.cc:654] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:47.701689 25089 log.cc:824] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:47.704984 25092 tablet_bootstrap.cc:492] T c86a0a8977b540db87048f7fd77a8e8b P fcdd78cc9b004378bf2531f78acafc78: Bootstrap starting.
I20260807 11:23:47.706156 25089 tablet_bootstrap.cc:492] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf: No bootstrap required, opened a new log
I20260807 11:23:47.706529 25089 ts_tablet_manager.cc:1397] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.006s
I20260807 11:23:47.709511 25092 tablet_bootstrap.cc:654] T c86a0a8977b540db87048f7fd77a8e8b P fcdd78cc9b004378bf2531f78acafc78: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:47.711640 25092 log.cc:824] T c86a0a8977b540db87048f7fd77a8e8b P fcdd78cc9b004378bf2531f78acafc78: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:47.711982 25093 tablet_bootstrap.cc:492] T c86a0a8977b540db87048f7fd77a8e8b P 2fd50e45dcc24d218e3c7f4951e5bf86: Bootstrap starting.
I20260807 11:23:47.716892 25093 tablet_bootstrap.cc:654] T c86a0a8977b540db87048f7fd77a8e8b P 2fd50e45dcc24d218e3c7f4951e5bf86: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:47.716921 25092 tablet_bootstrap.cc:492] T c86a0a8977b540db87048f7fd77a8e8b P fcdd78cc9b004378bf2531f78acafc78: No bootstrap required, opened a new log
I20260807 11:23:47.717309 25092 ts_tablet_manager.cc:1397] T c86a0a8977b540db87048f7fd77a8e8b P fcdd78cc9b004378bf2531f78acafc78: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.008s
I20260807 11:23:47.718243 25093 log.cc:824] T c86a0a8977b540db87048f7fd77a8e8b P 2fd50e45dcc24d218e3c7f4951e5bf86: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:47.721236 25089 raft_consensus.cc:348] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcdd78cc9b004378bf2531f78acafc78" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38243 } } peers { permanent_uuid: "af5f0cfe2e664d478c15b24cb6071baf" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 41871 } } peers { permanent_uuid: "2fd50e45dcc24d218e3c7f4951e5bf86" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39535 } }
I20260807 11:23:47.721801 25089 raft_consensus.cc:374] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:47.721937 25089 raft_consensus.cc:729] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af5f0cfe2e664d478c15b24cb6071baf, State: Initialized, Role: FOLLOWER
I20260807 11:23:47.722407 25093 tablet_bootstrap.cc:492] T c86a0a8977b540db87048f7fd77a8e8b P 2fd50e45dcc24d218e3c7f4951e5bf86: No bootstrap required, opened a new log
I20260807 11:23:47.722723 25093 ts_tablet_manager.cc:1397] T c86a0a8977b540db87048f7fd77a8e8b P 2fd50e45dcc24d218e3c7f4951e5bf86: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260807 11:23:47.722688 25089 consensus_queue.cc:260] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcdd78cc9b004378bf2531f78acafc78" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38243 } } peers { permanent_uuid: "af5f0cfe2e664d478c15b24cb6071baf" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 41871 } } peers { permanent_uuid: "2fd50e45dcc24d218e3c7f4951e5bf86" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39535 } }
I20260807 11:23:47.724850 25089 ts_tablet_manager.cc:1428] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf: Time spent starting tablet: real 0.018s user 0.007s sys 0.011s
I20260807 11:23:47.725190 24903 heartbeater.cc:503] Master 127.16.197.62:33957 was elected leader, sending a full tablet report...
I20260807 11:23:47.730222 25092 raft_consensus.cc:348] T c86a0a8977b540db87048f7fd77a8e8b P fcdd78cc9b004378bf2531f78acafc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcdd78cc9b004378bf2531f78acafc78" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38243 } } peers { permanent_uuid: "af5f0cfe2e664d478c15b24cb6071baf" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 41871 } } peers { permanent_uuid: "2fd50e45dcc24d218e3c7f4951e5bf86" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39535 } }
I20260807 11:23:47.730764 25092 raft_consensus.cc:374] T c86a0a8977b540db87048f7fd77a8e8b P fcdd78cc9b004378bf2531f78acafc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:47.730913 25092 raft_consensus.cc:729] T c86a0a8977b540db87048f7fd77a8e8b P fcdd78cc9b004378bf2531f78acafc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcdd78cc9b004378bf2531f78acafc78, State: Initialized, Role: FOLLOWER
I20260807 11:23:47.731681 25092 consensus_queue.cc:260] T c86a0a8977b540db87048f7fd77a8e8b P fcdd78cc9b004378bf2531f78acafc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcdd78cc9b004378bf2531f78acafc78" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38243 } } peers { permanent_uuid: "af5f0cfe2e664d478c15b24cb6071baf" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 41871 } } peers { permanent_uuid: "2fd50e45dcc24d218e3c7f4951e5bf86" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39535 } }
I20260807 11:23:47.734233 25092 ts_tablet_manager.cc:1428] T c86a0a8977b540db87048f7fd77a8e8b P fcdd78cc9b004378bf2531f78acafc78: Time spent starting tablet: real 0.017s user 0.014s sys 0.004s
I20260807 11:23:47.735692 25093 raft_consensus.cc:348] T c86a0a8977b540db87048f7fd77a8e8b P 2fd50e45dcc24d218e3c7f4951e5bf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcdd78cc9b004378bf2531f78acafc78" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38243 } } peers { permanent_uuid: "af5f0cfe2e664d478c15b24cb6071baf" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 41871 } } peers { permanent_uuid: "2fd50e45dcc24d218e3c7f4951e5bf86" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39535 } }
I20260807 11:23:47.736230 25093 raft_consensus.cc:374] T c86a0a8977b540db87048f7fd77a8e8b P 2fd50e45dcc24d218e3c7f4951e5bf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:47.736382 25093 raft_consensus.cc:729] T c86a0a8977b540db87048f7fd77a8e8b P 2fd50e45dcc24d218e3c7f4951e5bf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fd50e45dcc24d218e3c7f4951e5bf86, State: Initialized, Role: FOLLOWER
I20260807 11:23:47.737268 25093 consensus_queue.cc:260] T c86a0a8977b540db87048f7fd77a8e8b P 2fd50e45dcc24d218e3c7f4951e5bf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcdd78cc9b004378bf2531f78acafc78" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38243 } } peers { permanent_uuid: "af5f0cfe2e664d478c15b24cb6071baf" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 41871 } } peers { permanent_uuid: "2fd50e45dcc24d218e3c7f4951e5bf86" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39535 } }
I20260807 11:23:47.740199 25056 heartbeater.cc:503] Master 127.16.197.62:33957 was elected leader, sending a full tablet report...
I20260807 11:23:47.740309 25093 ts_tablet_manager.cc:1428] T c86a0a8977b540db87048f7fd77a8e8b P 2fd50e45dcc24d218e3c7f4951e5bf86: Time spent starting tablet: real 0.017s user 0.014s sys 0.004s
W20260807 11:23:47.765972 24749 tablet.cc:2369] T c86a0a8977b540db87048f7fd77a8e8b P fcdd78cc9b004378bf2531f78acafc78: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 11:23:47.794499 25063 tablet.cc:2369] T c86a0a8977b540db87048f7fd77a8e8b P 2fd50e45dcc24d218e3c7f4951e5bf86: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 11:23:47.905561 24913 tablet.cc:2369] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:23:47.966279 25096 raft_consensus.cc:482] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:23:47.966792 25096 raft_consensus.cc:504] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcdd78cc9b004378bf2531f78acafc78" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38243 } } peers { permanent_uuid: "af5f0cfe2e664d478c15b24cb6071baf" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 41871 } } peers { permanent_uuid: "2fd50e45dcc24d218e3c7f4951e5bf86" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39535 } }
I20260807 11:23:47.969035 25096 leader_election.cc:288] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcdd78cc9b004378bf2531f78acafc78 (127.16.197.1:38243), 2fd50e45dcc24d218e3c7f4951e5bf86 (127.16.197.3:39535)
I20260807 11:23:47.979357 25084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:47.969443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:51375 (local address 127.16.197.1:38243)
0807 11:23:47.969781 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:47.969788 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:47.971030 (+ 1242us) server_negotiation.cc:407] Connection header received
0807 11:23:47.971359 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:47.971363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:47.971475 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:47.971586 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:47.973022 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.973954 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.976807 (+ 2853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.978032 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.978303 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:47.978425 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:47.978734 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 11:23:47.978868 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 11:23:47.979868 25100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:47.969216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:38243 (local address 127.16.197.2:51375)
0807 11:23:47.970846 (+ 1630us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:47.970884 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:47.971181 (+ 297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:47.971608 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:47.971613 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:47.971637 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:47.972847 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:47.972864 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.974147 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.974151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:47.976567 (+ 2416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:47.976583 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.978401 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.978408 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:47.978568 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:47.978593 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 11:23:47.978706 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 11:23:47.978855 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1430,"mutex_wait_us":26,"spinlock_wait_cycles":13824,"thread_start_us":1396,"threads_started":1}
I20260807 11:23:47.980159 25088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:47.969626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:49635 (local address 127.16.197.3:39535)
0807 11:23:47.969861 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:47.969867 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:47.971172 (+ 1305us) server_negotiation.cc:407] Connection header received
0807 11:23:47.971848 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:47.971867 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:47.972015 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:47.972151 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:47.973373 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.974186 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.977148 (+ 2962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.978491 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.978748 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:47.978881 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:47.979384 (+ 503us) server_negotiation.cc:299] Negotiation successful
0807 11:23:47.979824 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260807 11:23:47.980512 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:47.969273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:39535 (local address 127.16.197.2:49635)
0807 11:23:47.971034 (+ 1761us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:47.971058 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:47.971209 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:47.972159 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:47.972163 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:47.972187 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:47.973152 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:47.973164 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.974334 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.974340 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:47.976916 (+ 2576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:47.976930 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:47.978939 (+ 2009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:47.978947 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:47.979205 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:47.979230 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 11:23:47.979346 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 11:23:47.979486 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1568,"mutex_wait_us":26,"spinlock_wait_cycles":2432,"thread_start_us":1143,"threads_started":1}
I20260807 11:23:47.981776 24697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c86a0a8977b540db87048f7fd77a8e8b" candidate_uuid: "af5f0cfe2e664d478c15b24cb6071baf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcdd78cc9b004378bf2531f78acafc78" is_pre_election: true
I20260807 11:23:47.981861 25006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c86a0a8977b540db87048f7fd77a8e8b" candidate_uuid: "af5f0cfe2e664d478c15b24cb6071baf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd50e45dcc24d218e3c7f4951e5bf86" is_pre_election: true
I20260807 11:23:47.982524 24697 raft_consensus.cc:2454] T c86a0a8977b540db87048f7fd77a8e8b P fcdd78cc9b004378bf2531f78acafc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af5f0cfe2e664d478c15b24cb6071baf in term 0.
I20260807 11:23:47.982580 25006 raft_consensus.cc:2454] T c86a0a8977b540db87048f7fd77a8e8b P 2fd50e45dcc24d218e3c7f4951e5bf86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af5f0cfe2e664d478c15b24cb6071baf in term 0.
I20260807 11:23:47.983510 24782 leader_election.cc:302] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2fd50e45dcc24d218e3c7f4951e5bf86, af5f0cfe2e664d478c15b24cb6071baf, fcdd78cc9b004378bf2531f78acafc78; no voters:
I20260807 11:23:47.984200 25096 raft_consensus.cc:2781] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:23:47.984539 25096 raft_consensus.cc:482] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:23:47.984748 25096 raft_consensus.cc:3037] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:47.988766 25096 raft_consensus.cc:504] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcdd78cc9b004378bf2531f78acafc78" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38243 } } peers { permanent_uuid: "af5f0cfe2e664d478c15b24cb6071baf" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 41871 } } peers { permanent_uuid: "2fd50e45dcc24d218e3c7f4951e5bf86" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39535 } }
I20260807 11:23:47.989596 25096 leader_election.cc:288] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf [CANDIDATE]: Term 1 election: Requested vote from peers fcdd78cc9b004378bf2531f78acafc78 (127.16.197.1:38243), 2fd50e45dcc24d218e3c7f4951e5bf86 (127.16.197.3:39535)
I20260807 11:23:47.990232 24697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c86a0a8977b540db87048f7fd77a8e8b" candidate_uuid: "af5f0cfe2e664d478c15b24cb6071baf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcdd78cc9b004378bf2531f78acafc78"
I20260807 11:23:47.990305 25006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c86a0a8977b540db87048f7fd77a8e8b" candidate_uuid: "af5f0cfe2e664d478c15b24cb6071baf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd50e45dcc24d218e3c7f4951e5bf86"
I20260807 11:23:47.990552 24697 raft_consensus.cc:3037] T c86a0a8977b540db87048f7fd77a8e8b P fcdd78cc9b004378bf2531f78acafc78 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:47.990612 25006 raft_consensus.cc:3037] T c86a0a8977b540db87048f7fd77a8e8b P 2fd50e45dcc24d218e3c7f4951e5bf86 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:47.994046 24697 raft_consensus.cc:2454] T c86a0a8977b540db87048f7fd77a8e8b P fcdd78cc9b004378bf2531f78acafc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af5f0cfe2e664d478c15b24cb6071baf in term 1.
I20260807 11:23:47.994482 25006 raft_consensus.cc:2454] T c86a0a8977b540db87048f7fd77a8e8b P 2fd50e45dcc24d218e3c7f4951e5bf86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af5f0cfe2e664d478c15b24cb6071baf in term 1.
I20260807 11:23:47.994699 24782 leader_election.cc:302] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af5f0cfe2e664d478c15b24cb6071baf, fcdd78cc9b004378bf2531f78acafc78; no voters:
I20260807 11:23:47.995172 25096 raft_consensus.cc:2781] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:47.996121 25096 raft_consensus.cc:686] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf [term 1 LEADER]: Becoming Leader. State: Replica: af5f0cfe2e664d478c15b24cb6071baf, State: Running, Role: LEADER
I20260807 11:23:47.996915 25096 consensus_queue.cc:237] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcdd78cc9b004378bf2531f78acafc78" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38243 } } peers { permanent_uuid: "af5f0cfe2e664d478c15b24cb6071baf" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 41871 } } peers { permanent_uuid: "2fd50e45dcc24d218e3c7f4951e5bf86" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39535 } }
I20260807 11:23:48.003836 23704 catalog_manager.cc:5495] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf reported cstate change: term changed from 0 to 1, leader changed from <none> to af5f0cfe2e664d478c15b24cb6071baf (127.16.197.2). New cstate: current_term: 1 leader_uuid: "af5f0cfe2e664d478c15b24cb6071baf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcdd78cc9b004378bf2531f78acafc78" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af5f0cfe2e664d478c15b24cb6071baf" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 41871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fd50e45dcc24d218e3c7f4951e5bf86" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39535 } health_report { overall_health: UNKNOWN } } }
I20260807 11:23:48.033223 25086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:48.026244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44362 (local address 127.16.197.2:41871)
0807 11:23:48.026468 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:48.026473 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:48.026491 (+ 18us) server_negotiation.cc:407] Connection header received
0807 11:23:48.026580 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:48.026584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:48.026696 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:48.026829 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:23:48.027514 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:48.028716 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:48.030513 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:48.031055 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:48.031154 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:48.031913 (+ 759us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:23:48.032556 (+ 643us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:23:48.032665 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:48.032782 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 11:23:48.032853 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 11:23:48.044557 25006 raft_consensus.cc:1261] T c86a0a8977b540db87048f7fd77a8e8b P 2fd50e45dcc24d218e3c7f4951e5bf86 [term 1 FOLLOWER]: Refusing update from remote peer af5f0cfe2e664d478c15b24cb6071baf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:48.044559 24697 raft_consensus.cc:1261] T c86a0a8977b540db87048f7fd77a8e8b P fcdd78cc9b004378bf2531f78acafc78 [term 1 FOLLOWER]: Refusing update from remote peer af5f0cfe2e664d478c15b24cb6071baf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:48.045811 25101 consensus_queue.cc:1035] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fd50e45dcc24d218e3c7f4951e5bf86" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:48.046501 25096 consensus_queue.cc:1035] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcdd78cc9b004378bf2531f78acafc78" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:48.080528 25105 mvcc.cc:205] Tried to move back new op lower bound from 7315873087656751104 to 7315873087479910400. Current Snapshot: MvccSnapshot[applied={T|T < 7315873087656751104}]
I20260807 11:23:48.093209 23704 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:23:48.093658 23704 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:23:48.106652 23704 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3f333150234643279ab25bac55bc7419: Sending DeleteTablet for 3 replicas of tablet c86a0a8977b540db87048f7fd77a8e8b
I20260807 11:23:48.108423 24833 tablet_service.cc:1514] Processing DeleteTablet for tablet c86a0a8977b540db87048f7fd77a8e8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:48 UTC) from {username='oryx', principal='oryx/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:44356
I20260807 11:23:48.108469 24676 tablet_service.cc:1514] Processing DeleteTablet for tablet c86a0a8977b540db87048f7fd77a8e8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:48 UTC) from {username='oryx', principal='oryx/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:51108
I20260807 11:23:48.112114 24985 tablet_service.cc:1514] Processing DeleteTablet for tablet c86a0a8977b540db87048f7fd77a8e8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:48 UTC) from {username='oryx', principal='oryx/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:42274
I20260807 11:23:48.131395 25117 tablet_replica.cc:330] T c86a0a8977b540db87048f7fd77a8e8b P 2fd50e45dcc24d218e3c7f4951e5bf86: stopping tablet replica
I20260807 11:23:48.132268 25117 raft_consensus.cc:2229] T c86a0a8977b540db87048f7fd77a8e8b P 2fd50e45dcc24d218e3c7f4951e5bf86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:48.133096 25117 raft_consensus.cc:2258] T c86a0a8977b540db87048f7fd77a8e8b P 2fd50e45dcc24d218e3c7f4951e5bf86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:48.141438 25117 ts_tablet_manager.cc:1905] T c86a0a8977b540db87048f7fd77a8e8b P 2fd50e45dcc24d218e3c7f4951e5bf86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:48.162213 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 23672
I20260807 11:23:48.163950 23737 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:48.163997 25117 ts_tablet_manager.cc:1918] T c86a0a8977b540db87048f7fd77a8e8b P 2fd50e45dcc24d218e3c7f4951e5bf86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:23:48.164296 25117 log.cc:1196] T c86a0a8977b540db87048f7fd77a8e8b P 2fd50e45dcc24d218e3c7f4951e5bf86: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/wal/wals/c86a0a8977b540db87048f7fd77a8e8b
I20260807 11:23:48.165462 25117 ts_tablet_manager.cc:1939] T c86a0a8977b540db87048f7fd77a8e8b P 2fd50e45dcc24d218e3c7f4951e5bf86: Deleting consensus metadata
I20260807 11:23:48.168397 25115 tablet_replica.cc:330] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf: stopping tablet replica
I20260807 11:23:48.168883 25115 raft_consensus.cc:2229] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf [term 1 LEADER]: Raft consensus shutting down.
I20260807 11:23:48.169691 25115 raft_consensus.cc:2258] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:48.172441 25115 ts_tablet_manager.cc:1905] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:48.172518 25114 tablet_replica.cc:330] T c86a0a8977b540db87048f7fd77a8e8b P fcdd78cc9b004378bf2531f78acafc78: stopping tablet replica
I20260807 11:23:48.173064 25114 raft_consensus.cc:2229] T c86a0a8977b540db87048f7fd77a8e8b P fcdd78cc9b004378bf2531f78acafc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:48.173838 25114 raft_consensus.cc:2258] T c86a0a8977b540db87048f7fd77a8e8b P fcdd78cc9b004378bf2531f78acafc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:48.181842 25114 ts_tablet_manager.cc:1905] T c86a0a8977b540db87048f7fd77a8e8b P fcdd78cc9b004378bf2531f78acafc78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:48.184312 25115 ts_tablet_manager.cc:1918] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:23:48.184725 25115 log.cc:1196] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/wal/wals/c86a0a8977b540db87048f7fd77a8e8b
I20260807 11:23:48.185923 25115 ts_tablet_manager.cc:1939] T c86a0a8977b540db87048f7fd77a8e8b P af5f0cfe2e664d478c15b24cb6071baf: Deleting consensus metadata
I20260807 11:23:48.193358 25114 ts_tablet_manager.cc:1918] T c86a0a8977b540db87048f7fd77a8e8b P fcdd78cc9b004378bf2531f78acafc78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:23:48.193612 25114 log.cc:1196] T c86a0a8977b540db87048f7fd77a8e8b P fcdd78cc9b004378bf2531f78acafc78: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/wal/wals/c86a0a8977b540db87048f7fd77a8e8b
I20260807 11:23:48.194459 25114 ts_tablet_manager.cc:1939] T c86a0a8977b540db87048f7fd77a8e8b P fcdd78cc9b004378bf2531f78acafc78: Deleting consensus metadata
I20260807 11:23:48.205190 25119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:48.115714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47532 (local address 127.16.197.61:41937)
0807 11:23:48.159005 (+ 43291us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:48.159013 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:48.159037 (+ 24us) server_negotiation.cc:407] Connection header received
0807 11:23:48.159109 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:48.159115 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:48.159227 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:48.159351 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:48.175629 (+ 16278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:48.177392 (+ 1763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:48.184371 (+ 6979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:48.185451 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:48.185584 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:48.198009 (+ 12425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:48.198016 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:48.198020 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:48.198058 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:48.201541 (+ 3483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:48.202326 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:48.202331 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:48.202333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:48.202482 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:48.203061 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:48.203066 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:48.203069 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:48.203483 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:48.203840 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:48.204355 (+ 515us) server_negotiation.cc:299] Negotiation successful
0807 11:23:48.204672 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43054,"thread_start_us":43777,"threads_started":1}
I20260807 11:23:48.227954 25120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:48.151740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46136 (local address 127.16.197.60:32811)
0807 11:23:48.174804 (+ 23064us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:48.174822 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:48.205334 (+ 30512us) server_negotiation.cc:407] Connection header received
0807 11:23:48.205589 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:48.205596 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:48.205689 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:48.207280 (+ 1591us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:48.210659 (+ 3379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:48.212939 (+ 2280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:48.215256 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:48.215926 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:48.216085 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:48.220081 (+ 3996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:48.220087 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:48.220090 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:48.220129 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:48.224377 (+ 4248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:48.225459 (+ 1082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:48.225464 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:48.225468 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:48.225624 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:48.226193 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:48.226199 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:48.226202 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:48.226520 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:48.226839 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:48.227219 (+ 380us) server_negotiation.cc:299] Negotiation successful
0807 11:23:48.227475 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22894,"thread_start_us":23063,"threads_started":1}
I20260807 11:23:48.364457 23688 catalog_manager.cc:4841] TS fcdd78cc9b004378bf2531f78acafc78 (127.16.197.1:38243): tablet c86a0a8977b540db87048f7fd77a8e8b (table test-table [id=e67b1a84945a406887249a3edf6cf051]) successfully deleted
I20260807 11:23:48.366092 23691 catalog_manager.cc:4841] TS 2fd50e45dcc24d218e3c7f4951e5bf86 (127.16.197.3:39535): tablet c86a0a8977b540db87048f7fd77a8e8b (table test-table [id=e67b1a84945a406887249a3edf6cf051]) successfully deleted
W20260807 11:23:48.366503 23691 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.16.197.3:39535, user_credentials={real_user=slave}} blocked reactor thread for 198011us
I20260807 11:23:48.366986 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 23672
I20260807 11:23:48.417884 25119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:48.397486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60458 (local address 127.16.197.61:41937)
0807 11:23:48.397727 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:48.397734 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:48.399328 (+ 1594us) server_negotiation.cc:407] Connection header received
0807 11:23:48.399632 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:48.399640 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:48.399745 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:48.399835 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:48.402438 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:48.404016 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:48.405423 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:48.405839 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:48.405979 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:48.409296 (+ 3317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:48.409301 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:48.409304 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:48.409344 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:48.413946 (+ 4602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:48.414846 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:48.414852 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:48.414856 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:48.415089 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:48.415524 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:48.415529 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:48.415532 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:48.415983 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:48.416271 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:48.417201 (+ 930us) server_negotiation.cc:299] Negotiation successful
0807 11:23:48.417426 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 11:23:48.420675 25120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:48.399433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60530 (local address 127.16.197.60:32811)
0807 11:23:48.399697 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:48.399703 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:48.399724 (+ 21us) server_negotiation.cc:407] Connection header received
0807 11:23:48.399837 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:48.399843 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:48.399922 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:48.400031 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:48.404817 (+ 4786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:48.406241 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:48.408189 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:48.409140 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:48.409265 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:48.412576 (+ 3311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:48.412584 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:48.412593 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:48.412653 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:48.415982 (+ 3329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:48.416618 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:48.416626 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:48.416632 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:48.416797 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:48.417441 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:48.417446 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:48.417450 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:48.418119 (+ 669us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:48.418563 (+ 444us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:48.419971 (+ 1408us) server_negotiation.cc:299] Negotiation successful
0807 11:23:48.420255 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 11:23:49.080063 25128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:49.079216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.16.197.1:52729)
0807 11:23:49.079732 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:49.079866 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":217,"threads_started":1}
W20260807 11:23:49.080572 24747 heartbeater.cc:644] Failed to heartbeat to 127.16.197.62:33957 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
I20260807 11:23:49.085822 25129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:49.085098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.16.197.2:47379)
0807 11:23:49.085549 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:49.085663 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":232,"threads_started":1}
W20260807 11:23:49.086300 24903 heartbeater.cc:644] Failed to heartbeat to 127.16.197.62:33957 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
I20260807 11:23:49.093052 25131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:49.092158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.16.197.3:53513)
0807 11:23:49.092645 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:49.092796 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":232,"threads_started":1}
W20260807 11:23:49.093464 25056 heartbeater.cc:644] Failed to heartbeat to 127.16.197.62:33957 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
I20260807 11:23:49.605232 25132 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f333150234643279ab25bac55bc7419)
I20260807 11:23:49.605677 25133 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f333150234643279ab25bac55bc7419)
I20260807 11:23:49.605655 25132 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: false } }
I20260807 11:23:49.605901 25133 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: false } }
I20260807 11:23:49.606779 25133 leader_election.cc:288] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f333150234643279ab25bac55bc7419 (127.16.197.62:33957), b2544d96e5a14f06b63cc24304065247 (127.16.197.61:41937)
I20260807 11:23:49.616627 25132 leader_election.cc:288] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f333150234643279ab25bac55bc7419 (127.16.197.62:33957), d2d3da4008c443a6a7596fdac5b97711 (127.16.197.60:32811)
I20260807 11:23:49.617026 25138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:49.607923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.0.0.1:33334)
0807 11:23:49.616394 (+ 8471us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:49.616675 (+ 281us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8304,"threads_started":1}
W20260807 11:23:49.621330 24337 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
I20260807 11:23:49.621502 25134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:49.607044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.0.0.1:33320)
0807 11:23:49.620084 (+ 13040us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:49.620372 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12860,"thread_start_us":12689,"threads_started":1}
W20260807 11:23:49.621805 24537 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
I20260807 11:23:49.625653 25134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:49.625030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.0.0.1:33358)
0807 11:23:49.625301 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:49.625421 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
W20260807 11:23:49.626080 24537 leader_election.cc:334] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3f333150234643279ab25bac55bc7419 (127.16.197.62:33957): Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
I20260807 11:23:49.628088 25137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:49.627491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.0.0.1:33370)
0807 11:23:49.627686 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:49.627874 (+ 188us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260807 11:23:49.628307 24337 leader_election.cc:334] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3f333150234643279ab25bac55bc7419 (127.16.197.62:33957): Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
I20260807 11:23:49.630532 25136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:49.607281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60462 (local address 127.16.197.61:41937)
0807 11:23:49.619424 (+ 12143us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:49.619433 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:49.621681 (+ 2248us) server_negotiation.cc:407] Connection header received
0807 11:23:49.622001 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:49.622010 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:49.622109 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:49.622193 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:49.623293 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:49.624963 (+ 1670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:49.628433 (+ 3470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:49.629593 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:49.629698 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:49.629767 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:49.630133 (+ 366us) server_negotiation.cc:299] Negotiation successful
0807 11:23:49.630235 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11967,"thread_start_us":12109,"threads_started":1}
I20260807 11:23:49.630568 25135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:49.607061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:41937 (local address 127.0.0.1:60462)
0807 11:23:49.621561 (+ 14500us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:49.621580 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:49.621797 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:49.622536 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:49.622544 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:49.622579 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:49.623022 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:49.623038 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:49.625237 (+ 2199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:49.625245 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:49.628243 (+ 2998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:49.628258 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:49.629754 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:49.629760 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:49.629945 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:49.629987 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 11:23:49.630088 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 11:23:49.630226 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14399,"thread_start_us":14238,"threads_started":1}
I20260807 11:23:49.631349 24362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d2d3da4008c443a6a7596fdac5b97711" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b2544d96e5a14f06b63cc24304065247" is_pre_election: true
I20260807 11:23:49.631659 24362 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2d3da4008c443a6a7596fdac5b97711 in term 1.
I20260807 11:23:49.632156 24534 leader_election.cc:302] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b2544d96e5a14f06b63cc24304065247, d2d3da4008c443a6a7596fdac5b97711; no voters: 3f333150234643279ab25bac55bc7419
I20260807 11:23:49.632547 25133 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 11:23:49.632704 25133 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3f333150234643279ab25bac55bc7419)
I20260807 11:23:49.632823 25133 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [term 1 FOLLOWER]: Advancing to term 2
I20260807 11:23:49.633692 25138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:49.614197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:32811 (local address 127.0.0.1:60538)
0807 11:23:49.617257 (+ 3060us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:49.617282 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:49.617466 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:49.626954 (+ 9488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:49.626986 (+ 32us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:49.627022 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:49.627325 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:49.627336 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:49.629290 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:49.629294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:49.631248 (+ 1954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:49.631261 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:49.632823 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:49.632829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:49.633108 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:49.633172 (+ 64us) client_negotiation.cc:769] Sending connection context
0807 11:23:49.633264 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 11:23:49.633478 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2945,"thread_start_us":1866,"threads_started":1}
I20260807 11:23:49.633862 25140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:49.621617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60538 (local address 127.16.197.60:32811)
0807 11:23:49.626606 (+ 4989us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:49.626614 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:49.626635 (+ 21us) server_negotiation.cc:407] Connection header received
0807 11:23:49.626700 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:49.626706 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:49.626777 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:49.626893 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:49.627536 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:49.629151 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:49.631431 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:49.632633 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:49.632789 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:49.632925 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:49.633335 (+ 410us) server_negotiation.cc:299] Negotiation successful
0807 11:23:49.633486 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4843,"thread_start_us":4691,"threads_started":1}
I20260807 11:23:49.634479 24562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b2544d96e5a14f06b63cc24304065247" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d2d3da4008c443a6a7596fdac5b97711" is_pre_election: true
I20260807 11:23:49.635740 25133 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: false } }
I20260807 11:23:49.636289 24562 raft_consensus.cc:2379] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2544d96e5a14f06b63cc24304065247 in current term 2: Already voted for candidate d2d3da4008c443a6a7596fdac5b97711 in this term.
I20260807 11:23:49.636483 25133 leader_election.cc:288] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [CANDIDATE]: Term 2 election: Requested vote from peers 3f333150234643279ab25bac55bc7419 (127.16.197.62:33957), b2544d96e5a14f06b63cc24304065247 (127.16.197.61:41937)
I20260807 11:23:49.637045 25135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:49.636527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.0.0.1:33382)
0807 11:23:49.636755 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:49.636859 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
I20260807 11:23:49.636991 24362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d2d3da4008c443a6a7596fdac5b97711" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b2544d96e5a14f06b63cc24304065247"
I20260807 11:23:49.637216 24362 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [term 1 FOLLOWER]: Advancing to term 2
I20260807 11:23:49.638188 25135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:49.637766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.0.0.1:33384)
0807 11:23:49.637926 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:49.638029 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 11:23:49.638115 24333 leader_election.cc:302] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2544d96e5a14f06b63cc24304065247; no voters: 3f333150234643279ab25bac55bc7419, d2d3da4008c443a6a7596fdac5b97711
W20260807 11:23:49.638432 24537 leader_election.cc:334] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3f333150234643279ab25bac55bc7419 (127.16.197.62:33957): Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
I20260807 11:23:49.639860 24362 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2d3da4008c443a6a7596fdac5b97711 in term 2.
I20260807 11:23:49.640089 25132 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 11:23:49.640327 24534 leader_election.cc:302] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b2544d96e5a14f06b63cc24304065247, d2d3da4008c443a6a7596fdac5b97711; no voters: 3f333150234643279ab25bac55bc7419
I20260807 11:23:49.640668 25133 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 11:23:49.641153 25133 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [term 2 LEADER]: Becoming Leader. State: Replica: d2d3da4008c443a6a7596fdac5b97711, State: Running, Role: LEADER
I20260807 11:23:49.641538 25133 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: false } }
I20260807 11:23:49.643392 25143 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2d3da4008c443a6a7596fdac5b97711. Latest consensus state: current_term: 2 leader_uuid: "d2d3da4008c443a6a7596fdac5b97711" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: false } } }
I20260807 11:23:49.643699 25143 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:49.644280 25145 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:23:49.648880 25145 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:23:49.650014 25145 catalog_manager.cc:1259] Loaded cluster ID: 9ded344e6a0f4ebf8ffae9aeddeaab78
I20260807 11:23:49.650130 25145 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:23:49.651634 25145 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:23:49.652691 25145 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711: Loaded TSK: 0
I20260807 11:23:49.653963 25145 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:23:49.675171 24553 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60530:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 11:23:49.682529 24553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:23:49.686793 25135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:49.686377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.0.0.1:33408)
0807 11:23:49.686547 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:49.686654 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 11:23:49.686878 24362 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [term 2 FOLLOWER]: Refusing update from remote peer d2d3da4008c443a6a7596fdac5b97711: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260807 11:23:49.687879 25135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:49.687493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.0.0.1:33422)
0807 11:23:49.687620 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:49.687685 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 11:23:49.688154 24537 consensus_peers.cc:487] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 -> Peer 3f333150234643279ab25bac55bc7419 (127.16.197.62:33957): Couldn't send request to peer 3f333150234643279ab25bac55bc7419. Status: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 11:23:49.688261 25143 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260807 11:23:49.691125 25132 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2d3da4008c443a6a7596fdac5b97711. Latest consensus state: current_term: 2 leader_uuid: "d2d3da4008c443a6a7596fdac5b97711" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: false } } }
I20260807 11:23:49.691418 25132 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 11:23:49.693327 25143 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "d2d3da4008c443a6a7596fdac5b97711" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: false } } }
I20260807 11:23:49.693652 25143 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:49.693670 25132 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "d2d3da4008c443a6a7596fdac5b97711" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3f333150234643279ab25bac55bc7419" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33957 } } peers { permanent_uuid: "b2544d96e5a14f06b63cc24304065247" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 41937 } attrs { promote: false } } peers { permanent_uuid: "d2d3da4008c443a6a7596fdac5b97711" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 32811 } attrs { promote: false } } }
I20260807 11:23:49.693961 25132 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2544d96e5a14f06b63cc24304065247 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 11:23:49.724792 25135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:49.701912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:39535 (local address 127.0.0.1:60534)
0807 11:23:49.702081 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:49.702101 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:49.702217 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:49.719014 (+ 16797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:49.719021 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:49.719055 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:49.719275 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:49.719286 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:49.720425 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:49.720430 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:49.722234 (+ 1804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:49.722246 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:49.724098 (+ 1852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:49.724104 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:49.724279 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:49.724310 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 11:23:49.724384 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 11:23:49.724493 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":1}
I20260807 11:23:49.725204 25153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:49.702863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:41871 (local address 127.0.0.1:39658)
0807 11:23:49.716244 (+ 13381us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:49.716279 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:49.716832 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:49.717833 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:49.717840 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:49.717874 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:49.718291 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:49.718305 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:49.720009 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:49.720013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:49.722368 (+ 2355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:49.722383 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:49.724319 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:49.724329 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:49.724527 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:49.724562 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 11:23:49.724657 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 11:23:49.724801 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13197,"mutex_wait_us":19,"thread_start_us":12981,"threads_started":1}
I20260807 11:23:49.725665 25152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:49.702312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60534 (local address 127.16.197.3:39535)
0807 11:23:49.718288 (+ 15976us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:49.718296 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:49.718316 (+ 20us) server_negotiation.cc:407] Connection header received
0807 11:23:49.718394 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:49.718402 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:49.718489 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:49.719015 (+ 526us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:49.719433 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:49.720234 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:49.722534 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:49.723914 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:49.725058 (+ 1144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:49.725175 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:49.725233 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 11:23:49.725378 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15826,"thread_start_us":16493,"threads_started":1}
I20260807 11:23:49.725652 25154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:49.703111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39658 (local address 127.16.197.2:41871)
0807 11:23:49.715543 (+ 12432us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:49.715555 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:49.716662 (+ 1107us) server_negotiation.cc:407] Connection header received
0807 11:23:49.717039 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:49.717049 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:49.717160 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:49.717461 (+ 301us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:49.718503 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:49.719818 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:49.722570 (+ 2752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:49.723842 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:49.723965 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:49.724114 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:49.724717 (+ 603us) server_negotiation.cc:299] Negotiation successful
0807 11:23:49.725272 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12209,"thread_start_us":12443,"threads_started":1}
I20260807 11:23:49.726850 25134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:49.702365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:38243 (local address 127.0.0.1:57568)
0807 11:23:49.702517 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:49.702545 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:49.702806 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:49.718709 (+ 15903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:49.718714 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:49.718740 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:49.719108 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:49.719120 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:49.721733 (+ 2613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:49.721739 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:49.723819 (+ 2080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:49.723835 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:49.726103 (+ 2268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:49.726112 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:49.726276 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:49.726308 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 11:23:49.726420 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 11:23:49.726570 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":1}
I20260807 11:23:49.726953 25155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:49.703121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57568 (local address 127.16.197.1:38243)
0807 11:23:49.718318 (+ 15197us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:49.718323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:49.718341 (+ 18us) server_negotiation.cc:407] Connection header received
0807 11:23:49.718406 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:49.718410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:49.718466 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:49.718688 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:49.719309 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:49.721571 (+ 2262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:49.724059 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:49.725663 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:49.726003 (+ 340us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:49.726111 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:49.726444 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 11:23:49.726598 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15063,"thread_start_us":15665,"threads_started":1}
I20260807 11:23:49.727285 24833 tablet_service.cc:1467] Processing CreateTablet for tablet 69c7a52d4ec24514b6694cd657ecb9b3 (DEFAULT_TABLE table=test-table [id=bc63ab011ee24a679e22d77017f3ff06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:49.727592 24985 tablet_service.cc:1467] Processing CreateTablet for tablet 69c7a52d4ec24514b6694cd657ecb9b3 (DEFAULT_TABLE table=test-table [id=bc63ab011ee24a679e22d77017f3ff06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:49.727819 24833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69c7a52d4ec24514b6694cd657ecb9b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:49.727743 24676 tablet_service.cc:1467] Processing CreateTablet for tablet 69c7a52d4ec24514b6694cd657ecb9b3 (DEFAULT_TABLE table=test-table [id=bc63ab011ee24a679e22d77017f3ff06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:49.727960 24985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69c7a52d4ec24514b6694cd657ecb9b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:49.728178 24676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69c7a52d4ec24514b6694cd657ecb9b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:49.733925 25156 tablet_bootstrap.cc:492] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86: Bootstrap starting.
I20260807 11:23:49.735662 25156 tablet_bootstrap.cc:654] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:49.738354 25156 tablet_bootstrap.cc:492] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86: No bootstrap required, opened a new log
I20260807 11:23:49.738507 25156 ts_tablet_manager.cc:1397] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 11:23:49.739274 25156 raft_consensus.cc:348] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd50e45dcc24d218e3c7f4951e5bf86" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39535 } } peers { permanent_uuid: "fcdd78cc9b004378bf2531f78acafc78" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38243 } } peers { permanent_uuid: "af5f0cfe2e664d478c15b24cb6071baf" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 41871 } }
I20260807 11:23:49.739485 25156 raft_consensus.cc:374] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:49.739575 25156 raft_consensus.cc:729] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fd50e45dcc24d218e3c7f4951e5bf86, State: Initialized, Role: FOLLOWER
I20260807 11:23:49.739900 25159 tablet_bootstrap.cc:492] T 69c7a52d4ec24514b6694cd657ecb9b3 P af5f0cfe2e664d478c15b24cb6071baf: Bootstrap starting.
I20260807 11:23:49.739858 25156 consensus_queue.cc:260] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd50e45dcc24d218e3c7f4951e5bf86" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39535 } } peers { permanent_uuid: "fcdd78cc9b004378bf2531f78acafc78" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38243 } } peers { permanent_uuid: "af5f0cfe2e664d478c15b24cb6071baf" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 41871 } }
I20260807 11:23:49.744077 25156 ts_tablet_manager.cc:1428] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260807 11:23:49.744375 25055 heartbeater.cc:503] Master 127.16.197.60:32811 was elected leader, sending a full tablet report...
I20260807 11:23:49.745802 25159 tablet_bootstrap.cc:654] T 69c7a52d4ec24514b6694cd657ecb9b3 P af5f0cfe2e664d478c15b24cb6071baf: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:49.751965 25162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:49.741039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.16.197.3:46081)
0807 11:23:49.751501 (+ 10462us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:49.751642 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10317,"thread_start_us":10165,"threads_started":1}
I20260807 11:23:49.752735 25158 tablet_bootstrap.cc:492] T 69c7a52d4ec24514b6694cd657ecb9b3 P fcdd78cc9b004378bf2531f78acafc78: Bootstrap starting.
I20260807 11:23:49.752754 25159 tablet_bootstrap.cc:492] T 69c7a52d4ec24514b6694cd657ecb9b3 P af5f0cfe2e664d478c15b24cb6071baf: No bootstrap required, opened a new log
I20260807 11:23:49.753058 25162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:49.752606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.16.197.3:45099)
0807 11:23:49.752836 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:49.752931 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260807 11:23:49.753049 25159 ts_tablet_manager.cc:1397] T 69c7a52d4ec24514b6694cd657ecb9b3 P af5f0cfe2e664d478c15b24cb6071baf: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.000s
I20260807 11:23:49.754110 25159 raft_consensus.cc:348] T 69c7a52d4ec24514b6694cd657ecb9b3 P af5f0cfe2e664d478c15b24cb6071baf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd50e45dcc24d218e3c7f4951e5bf86" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39535 } } peers { permanent_uuid: "fcdd78cc9b004378bf2531f78acafc78" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38243 } } peers { permanent_uuid: "af5f0cfe2e664d478c15b24cb6071baf" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 41871 } }
I20260807 11:23:49.754423 25159 raft_consensus.cc:374] T 69c7a52d4ec24514b6694cd657ecb9b3 P af5f0cfe2e664d478c15b24cb6071baf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:49.754563 25159 raft_consensus.cc:729] T 69c7a52d4ec24514b6694cd657ecb9b3 P af5f0cfe2e664d478c15b24cb6071baf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af5f0cfe2e664d478c15b24cb6071baf, State: Initialized, Role: FOLLOWER
I20260807 11:23:49.754899 25159 consensus_queue.cc:260] T 69c7a52d4ec24514b6694cd657ecb9b3 P af5f0cfe2e664d478c15b24cb6071baf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd50e45dcc24d218e3c7f4951e5bf86" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39535 } } peers { permanent_uuid: "fcdd78cc9b004378bf2531f78acafc78" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38243 } } peers { permanent_uuid: "af5f0cfe2e664d478c15b24cb6071baf" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 41871 } }
I20260807 11:23:49.755205 25158 tablet_bootstrap.cc:654] T 69c7a52d4ec24514b6694cd657ecb9b3 P fcdd78cc9b004378bf2531f78acafc78: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:49.755986 25159 ts_tablet_manager.cc:1428] T 69c7a52d4ec24514b6694cd657ecb9b3 P af5f0cfe2e664d478c15b24cb6071baf: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 11:23:49.758163 24902 heartbeater.cc:503] Master 127.16.197.60:32811 was elected leader, sending a full tablet report...
I20260807 11:23:49.767823 25166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:49.756794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.16.197.2:46181)
0807 11:23:49.767448 (+ 10654us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:49.767592 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10539,"thread_start_us":10399,"threads_started":1}
I20260807 11:23:49.768996 25166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:49.768660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.16.197.2:40155)
0807 11:23:49.768815 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:49.768888 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 11:23:49.769493 25158 tablet_bootstrap.cc:492] T 69c7a52d4ec24514b6694cd657ecb9b3 P fcdd78cc9b004378bf2531f78acafc78: No bootstrap required, opened a new log
I20260807 11:23:49.769677 25158 ts_tablet_manager.cc:1397] T 69c7a52d4ec24514b6694cd657ecb9b3 P fcdd78cc9b004378bf2531f78acafc78: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.012s
I20260807 11:23:49.770380 25158 raft_consensus.cc:348] T 69c7a52d4ec24514b6694cd657ecb9b3 P fcdd78cc9b004378bf2531f78acafc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd50e45dcc24d218e3c7f4951e5bf86" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39535 } } peers { permanent_uuid: "fcdd78cc9b004378bf2531f78acafc78" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38243 } } peers { permanent_uuid: "af5f0cfe2e664d478c15b24cb6071baf" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 41871 } }
I20260807 11:23:49.770620 25158 raft_consensus.cc:374] T 69c7a52d4ec24514b6694cd657ecb9b3 P fcdd78cc9b004378bf2531f78acafc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:49.770747 25158 raft_consensus.cc:729] T 69c7a52d4ec24514b6694cd657ecb9b3 P fcdd78cc9b004378bf2531f78acafc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcdd78cc9b004378bf2531f78acafc78, State: Initialized, Role: FOLLOWER
I20260807 11:23:49.771098 25158 consensus_queue.cc:260] T 69c7a52d4ec24514b6694cd657ecb9b3 P fcdd78cc9b004378bf2531f78acafc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd50e45dcc24d218e3c7f4951e5bf86" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39535 } } peers { permanent_uuid: "fcdd78cc9b004378bf2531f78acafc78" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38243 } } peers { permanent_uuid: "af5f0cfe2e664d478c15b24cb6071baf" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 41871 } }
I20260807 11:23:49.771811 25158 ts_tablet_manager.cc:1428] T 69c7a52d4ec24514b6694cd657ecb9b3 P fcdd78cc9b004378bf2531f78acafc78: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 11:23:49.773264 25169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:49.772192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.16.197.1:44421)
0807 11:23:49.772821 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:49.773004 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":325,"threads_started":1}
I20260807 11:23:49.774326 24746 heartbeater.cc:503] Master 127.16.197.60:32811 was elected leader, sending a full tablet report...
I20260807 11:23:49.774442 25169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:49.774000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.16.197.1:41093)
0807 11:23:49.774169 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:49.774266 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 11:23:50.120333 25161 raft_consensus.cc:482] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:23:50.120618 25161 raft_consensus.cc:504] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd50e45dcc24d218e3c7f4951e5bf86" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39535 } } peers { permanent_uuid: "fcdd78cc9b004378bf2531f78acafc78" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38243 } } peers { permanent_uuid: "af5f0cfe2e664d478c15b24cb6071baf" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 41871 } }
I20260807 11:23:50.122409 25161 leader_election.cc:288] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcdd78cc9b004378bf2531f78acafc78 (127.16.197.1:38243), af5f0cfe2e664d478c15b24cb6071baf (127.16.197.2:41871)
I20260807 11:23:50.130331 25155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:50.122759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:56511 (local address 127.16.197.1:38243)
0807 11:23:50.123049 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:50.123055 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:50.123381 (+ 326us) server_negotiation.cc:407] Connection header received
0807 11:23:50.123708 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:50.123715 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:50.123804 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:50.123907 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:50.124435 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:50.125601 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:50.128536 (+ 2935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:50.129312 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:50.129506 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:50.129585 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:50.129944 (+ 359us) server_negotiation.cc:299] Negotiation successful
0807 11:23:50.130083 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 11:23:50.130342 25154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:50.123047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:52719 (local address 127.16.197.2:41871)
0807 11:23:50.123302 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:50.123311 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:50.123727 (+ 416us) server_negotiation.cc:407] Connection header received
0807 11:23:50.123873 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:50.123881 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:50.123960 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:50.124048 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:50.124659 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:50.125709 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:50.128491 (+ 2782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:50.129319 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:50.129508 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:50.129590 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:50.129930 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 11:23:50.130062 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 11:23:50.130501 25170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:50.122687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:41871 (local address 127.16.197.3:52719)
0807 11:23:50.123609 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:50.123633 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:50.123739 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:50.124081 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:50.124086 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:50.124103 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:50.124494 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:50.124503 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:50.125832 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:50.125838 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:50.128315 (+ 2477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:50.128331 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:50.129550 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:50.129557 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:50.129766 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:50.129791 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 11:23:50.129900 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 11:23:50.130057 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":792,"mutex_wait_us":79,"thread_start_us":588,"threads_started":1}
I20260807 11:23:50.130812 25162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:50.122639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:38243 (local address 127.16.197.3:56511)
0807 11:23:50.123011 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:50.123043 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:50.123513 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:50.123939 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:50.123943 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:50.123963 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:50.124286 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:50.124299 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:50.125744 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:50.125748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:50.128315 (+ 2567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:50.128331 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:50.129531 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:50.129537 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:50.129725 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:50.129798 (+ 73us) client_negotiation.cc:769] Sending connection context
0807 11:23:50.129905 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 11:23:50.130077 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":253}
I20260807 11:23:50.131528 24697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69c7a52d4ec24514b6694cd657ecb9b3" candidate_uuid: "2fd50e45dcc24d218e3c7f4951e5bf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcdd78cc9b004378bf2531f78acafc78" is_pre_election: true
I20260807 11:23:50.131538 24854 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69c7a52d4ec24514b6694cd657ecb9b3" candidate_uuid: "2fd50e45dcc24d218e3c7f4951e5bf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af5f0cfe2e664d478c15b24cb6071baf" is_pre_election: true
I20260807 11:23:50.131778 24697 raft_consensus.cc:2454] T 69c7a52d4ec24514b6694cd657ecb9b3 P fcdd78cc9b004378bf2531f78acafc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fd50e45dcc24d218e3c7f4951e5bf86 in term 0.
I20260807 11:23:50.132019 24854 raft_consensus.cc:2454] T 69c7a52d4ec24514b6694cd657ecb9b3 P af5f0cfe2e664d478c15b24cb6071baf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fd50e45dcc24d218e3c7f4951e5bf86 in term 0.
I20260807 11:23:50.132407 24934 leader_election.cc:302] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fd50e45dcc24d218e3c7f4951e5bf86, fcdd78cc9b004378bf2531f78acafc78; no voters:
I20260807 11:23:50.132881 25161 raft_consensus.cc:2781] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:23:50.133015 25161 raft_consensus.cc:482] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:23:50.133122 25161 raft_consensus.cc:3037] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:50.135430 25161 raft_consensus.cc:504] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd50e45dcc24d218e3c7f4951e5bf86" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39535 } } peers { permanent_uuid: "fcdd78cc9b004378bf2531f78acafc78" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38243 } } peers { permanent_uuid: "af5f0cfe2e664d478c15b24cb6071baf" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 41871 } }
I20260807 11:23:50.136061 25161 leader_election.cc:288] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86 [CANDIDATE]: Term 1 election: Requested vote from peers fcdd78cc9b004378bf2531f78acafc78 (127.16.197.1:38243), af5f0cfe2e664d478c15b24cb6071baf (127.16.197.2:41871)
I20260807 11:23:50.136538 24697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69c7a52d4ec24514b6694cd657ecb9b3" candidate_uuid: "2fd50e45dcc24d218e3c7f4951e5bf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcdd78cc9b004378bf2531f78acafc78"
I20260807 11:23:50.136595 24854 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69c7a52d4ec24514b6694cd657ecb9b3" candidate_uuid: "2fd50e45dcc24d218e3c7f4951e5bf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af5f0cfe2e664d478c15b24cb6071baf"
I20260807 11:23:50.136755 24697 raft_consensus.cc:3037] T 69c7a52d4ec24514b6694cd657ecb9b3 P fcdd78cc9b004378bf2531f78acafc78 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:50.136801 24854 raft_consensus.cc:3037] T 69c7a52d4ec24514b6694cd657ecb9b3 P af5f0cfe2e664d478c15b24cb6071baf [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:50.138828 24697 raft_consensus.cc:2454] T 69c7a52d4ec24514b6694cd657ecb9b3 P fcdd78cc9b004378bf2531f78acafc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fd50e45dcc24d218e3c7f4951e5bf86 in term 1.
I20260807 11:23:50.138911 24854 raft_consensus.cc:2454] T 69c7a52d4ec24514b6694cd657ecb9b3 P af5f0cfe2e664d478c15b24cb6071baf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fd50e45dcc24d218e3c7f4951e5bf86 in term 1.
I20260807 11:23:50.139340 24934 leader_election.cc:302] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fd50e45dcc24d218e3c7f4951e5bf86, fcdd78cc9b004378bf2531f78acafc78; no voters:
I20260807 11:23:50.139679 25161 raft_consensus.cc:2781] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:50.140156 25161 raft_consensus.cc:686] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86 [term 1 LEADER]: Becoming Leader. State: Replica: 2fd50e45dcc24d218e3c7f4951e5bf86, State: Running, Role: LEADER
I20260807 11:23:50.140656 25161 consensus_queue.cc:237] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd50e45dcc24d218e3c7f4951e5bf86" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39535 } } peers { permanent_uuid: "fcdd78cc9b004378bf2531f78acafc78" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38243 } } peers { permanent_uuid: "af5f0cfe2e664d478c15b24cb6071baf" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 41871 } }
I20260807 11:23:50.141396 25162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:50.140952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.16.197.3:50915)
0807 11:23:50.141146 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:50.141261 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 11:23:50.144532 24553 catalog_manager.cc:5495] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2fd50e45dcc24d218e3c7f4951e5bf86 (127.16.197.3). New cstate: current_term: 1 leader_uuid: "2fd50e45dcc24d218e3c7f4951e5bf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd50e45dcc24d218e3c7f4951e5bf86" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 39535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcdd78cc9b004378bf2531f78acafc78" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af5f0cfe2e664d478c15b24cb6071baf" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 41871 } health_report { overall_health: UNKNOWN } } }
I20260807 11:23:50.182507 25152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:50.176782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60546 (local address 127.16.197.3:39535)
0807 11:23:50.176996 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:50.177000 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:50.177358 (+ 358us) server_negotiation.cc:407] Connection header received
0807 11:23:50.177480 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:50.177483 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:50.177564 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:50.177745 (+ 181us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:23:50.178172 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:50.178945 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:50.180353 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:50.180770 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:50.180911 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:50.181332 (+ 421us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:23:50.181896 (+ 564us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:23:50.182058 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:50.182139 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 11:23:50.182223 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 11:23:50.186942 24697 raft_consensus.cc:1261] T 69c7a52d4ec24514b6694cd657ecb9b3 P fcdd78cc9b004378bf2531f78acafc78 [term 1 FOLLOWER]: Refusing update from remote peer 2fd50e45dcc24d218e3c7f4951e5bf86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:50.187412 24854 raft_consensus.cc:1261] T 69c7a52d4ec24514b6694cd657ecb9b3 P af5f0cfe2e664d478c15b24cb6071baf [term 1 FOLLOWER]: Refusing update from remote peer 2fd50e45dcc24d218e3c7f4951e5bf86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:50.188045 25161 consensus_queue.cc:1035] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcdd78cc9b004378bf2531f78acafc78" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:50.188405 25171 consensus_queue.cc:1035] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af5f0cfe2e664d478c15b24cb6071baf" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 41871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:50.191107 25166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:50.190495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.16.197.2:52441)
0807 11:23:50.190755 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:50.190877 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
I20260807 11:23:50.193188 25169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:50.192703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.16.197.1:51541)
0807 11:23:50.192865 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:50.193004 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 11:23:50.199568 25166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:50.198763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.16.197.2:36377)
0807 11:23:50.199160 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:50.199297 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232}
I20260807 11:23:50.202824 25162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:50.202310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.16.197.3:57675)
0807 11:23:50.202510 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:50.202630 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 11:23:50.212741 25169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:50.212119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.16.197.1:57731)
0807 11:23:50.212415 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:50.212526 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
I20260807 11:23:50.215134 25177 mvcc.cc:205] Tried to move back new op lower bound from 7315873096437497856 to 7315873096260358144. Current Snapshot: MvccSnapshot[applied={T|T < 7315873096437497856}]
I20260807 11:23:50.225443 25162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:50.224958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.16.197.3:60785)
0807 11:23:50.225136 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:50.225242 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 11:23:50.227991 24552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:23:50.228293 24552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:23:50.234176 24552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d2d3da4008c443a6a7596fdac5b97711: Sending DeleteTablet for 3 replicas of tablet 69c7a52d4ec24514b6694cd657ecb9b3
I20260807 11:23:50.235172 24985 tablet_service.cc:1514] Processing DeleteTablet for tablet 69c7a52d4ec24514b6694cd657ecb9b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:50 UTC) from {username='oryx', principal='oryx/127.16.197.60@KRBTEST.COM'} at 127.0.0.1:60534
I20260807 11:23:50.235569 24833 tablet_service.cc:1514] Processing DeleteTablet for tablet 69c7a52d4ec24514b6694cd657ecb9b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:50 UTC) from {username='oryx', principal='oryx/127.16.197.60@KRBTEST.COM'} at 127.0.0.1:39658
I20260807 11:23:50.235780 24676 tablet_service.cc:1514] Processing DeleteTablet for tablet 69c7a52d4ec24514b6694cd657ecb9b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:50 UTC) from {username='oryx', principal='oryx/127.16.197.60@KRBTEST.COM'} at 127.0.0.1:57568
I20260807 11:23:50.236967 25191 tablet_replica.cc:330] T 69c7a52d4ec24514b6694cd657ecb9b3 P fcdd78cc9b004378bf2531f78acafc78: stopping tablet replica
I20260807 11:23:50.237309 25191 raft_consensus.cc:2229] T 69c7a52d4ec24514b6694cd657ecb9b3 P fcdd78cc9b004378bf2531f78acafc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:50.237769 25191 raft_consensus.cc:2258] T 69c7a52d4ec24514b6694cd657ecb9b3 P fcdd78cc9b004378bf2531f78acafc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:50.238504 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 24524
I20260807 11:23:50.239202 25191 ts_tablet_manager.cc:1905] T 69c7a52d4ec24514b6694cd657ecb9b3 P fcdd78cc9b004378bf2531f78acafc78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:50.248648 25191 ts_tablet_manager.cc:1918] T 69c7a52d4ec24514b6694cd657ecb9b3 P fcdd78cc9b004378bf2531f78acafc78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:23:50.248979 25191 log.cc:1196] T 69c7a52d4ec24514b6694cd657ecb9b3 P fcdd78cc9b004378bf2531f78acafc78: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-0/wal/wals/69c7a52d4ec24514b6694cd657ecb9b3
I20260807 11:23:50.249699 25191 ts_tablet_manager.cc:1939] T 69c7a52d4ec24514b6694cd657ecb9b3 P fcdd78cc9b004378bf2531f78acafc78: Deleting consensus metadata
I20260807 11:23:50.251174 25190 tablet_replica.cc:330] T 69c7a52d4ec24514b6694cd657ecb9b3 P af5f0cfe2e664d478c15b24cb6071baf: stopping tablet replica
I20260807 11:23:50.251329 24534 catalog_manager.cc:4841] TS fcdd78cc9b004378bf2531f78acafc78 (127.16.197.1:38243): tablet 69c7a52d4ec24514b6694cd657ecb9b3 (table test-table [id=bc63ab011ee24a679e22d77017f3ff06]) successfully deleted
I20260807 11:23:50.251428 25190 raft_consensus.cc:2229] T 69c7a52d4ec24514b6694cd657ecb9b3 P af5f0cfe2e664d478c15b24cb6071baf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:50.251767 25190 raft_consensus.cc:2258] T 69c7a52d4ec24514b6694cd657ecb9b3 P af5f0cfe2e664d478c15b24cb6071baf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:50.252763 25190 ts_tablet_manager.cc:1905] T 69c7a52d4ec24514b6694cd657ecb9b3 P af5f0cfe2e664d478c15b24cb6071baf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:50.255414 25188 tablet_replica.cc:330] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86: stopping tablet replica
I20260807 11:23:50.255635 25188 raft_consensus.cc:2229] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86 [term 1 LEADER]: Raft consensus shutting down.
I20260807 11:23:50.256317 25188 raft_consensus.cc:2258] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:50.258891 25188 ts_tablet_manager.cc:1905] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:50.258976 25190 ts_tablet_manager.cc:1918] T 69c7a52d4ec24514b6694cd657ecb9b3 P af5f0cfe2e664d478c15b24cb6071baf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:23:50.259105 25190 log.cc:1196] T 69c7a52d4ec24514b6694cd657ecb9b3 P af5f0cfe2e664d478c15b24cb6071baf: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-1/wal/wals/69c7a52d4ec24514b6694cd657ecb9b3
I20260807 11:23:50.259568 25190 ts_tablet_manager.cc:1939] T 69c7a52d4ec24514b6694cd657ecb9b3 P af5f0cfe2e664d478c15b24cb6071baf: Deleting consensus metadata
I20260807 11:23:50.260691 24537 catalog_manager.cc:4841] TS af5f0cfe2e664d478c15b24cb6071baf (127.16.197.2:41871): tablet 69c7a52d4ec24514b6694cd657ecb9b3 (table test-table [id=bc63ab011ee24a679e22d77017f3ff06]) successfully deleted
I20260807 11:23:50.267536 25188 ts_tablet_manager.cc:1918] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:23:50.267690 25188 log.cc:1196] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786101765888975-17172-0/minicluster-data/ts-2/wal/wals/69c7a52d4ec24514b6694cd657ecb9b3
I20260807 11:23:50.268218 25188 ts_tablet_manager.cc:1939] T 69c7a52d4ec24514b6694cd657ecb9b3 P 2fd50e45dcc24d218e3c7f4951e5bf86: Deleting consensus metadata
I20260807 11:23:50.269447 24537 catalog_manager.cc:4841] TS 2fd50e45dcc24d218e3c7f4951e5bf86 (127.16.197.3:39535): tablet 69c7a52d4ec24514b6694cd657ecb9b3 (table test-table [id=bc63ab011ee24a679e22d77017f3ff06]) successfully deleted
I20260807 11:23:50.281369 25155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:50.239701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57578 (local address 127.16.197.1:38243)
0807 11:23:50.239912 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:50.239918 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:50.253886 (+ 13968us) server_negotiation.cc:407] Connection header received
0807 11:23:50.254155 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:50.254164 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:50.254271 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:50.254389 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:50.256870 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:50.258104 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:50.264941 (+ 6837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:50.265617 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:50.265824 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:50.274291 (+ 8467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:50.274311 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:50.274332 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:50.274389 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:50.278292 (+ 3903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:50.279210 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:50.279217 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:50.279219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:50.279376 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:50.279828 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:50.279831 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:50.279833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:50.280184 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:50.280519 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:50.280886 (+ 367us) server_negotiation.cc:299] Negotiation successful
0807 11:23:50.281054 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 11:23:50.282119 24740 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:50.303956 25194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:50.302939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.0.0.1:33434)
0807 11:23:50.303536 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:50.303683 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":255,"threads_started":1}
I20260807 11:23:50.305814 25194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:50.305313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.0.0.1:33440)
0807 11:23:50.305509 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:50.305637 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 11:23:50.388991 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 24524
I20260807 11:23:50.409571 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 24765
I20260807 11:23:50.423921 25154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:50.410430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39664 (local address 127.16.197.2:41871)
0807 11:23:50.410605 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:50.410614 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:50.410694 (+ 80us) server_negotiation.cc:407] Connection header received
0807 11:23:50.410901 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:50.410907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:50.411150 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:50.411258 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:50.412692 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:50.413443 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:50.414669 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:50.415164 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:50.415298 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:50.417928 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:50.417942 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:50.417958 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:50.417996 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:50.421029 (+ 3033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:50.421752 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:50.421760 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:50.421762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:50.421914 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:50.422299 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:50.422305 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:50.422308 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:50.422704 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:50.423038 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:50.423373 (+ 335us) server_negotiation.cc:299] Negotiation successful
0807 11:23:50.423612 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 11:23:50.424571 24897 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:50.564174 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 24765
I20260807 11:23:50.589964 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 24916
I20260807 11:23:50.607385 25152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:50.591011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60558 (local address 127.16.197.3:39535)
0807 11:23:50.591225 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:50.591232 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:50.591253 (+ 21us) server_negotiation.cc:407] Connection header received
0807 11:23:50.591321 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:50.591326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:50.591414 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:50.591549 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:50.592940 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:50.594157 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:50.595615 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:50.596317 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:50.596475 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:50.599060 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:50.599081 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:50.599108 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:50.599155 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:50.604117 (+ 4962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:50.604885 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:50.604898 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:50.604903 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:50.605125 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:50.605528 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:50.605534 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:50.605537 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:50.606073 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:50.606413 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:50.606637 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 11:23:50.606908 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 11:23:50.608297 25049 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:50.744983 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 24916
I20260807 11:23:50.769363 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 24203
I20260807 11:23:50.770474 24382 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:50.891906 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 24203
I20260807 11:23:50.908263 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:50.907428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.0.0.1:33442)
0807 11:23:50.907963 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:50.908062 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":302,"threads_started":1}
I20260807 11:23:50.909679 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:50.909315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33957 (local address 127.0.0.1:33444)
0807 11:23:50.909487 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:50.909557 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:33957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 11:23:50.915493 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 24309
I20260807 11:23:50.916512 24585 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:51.044540 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 24309
2026-08-07T11:23:51Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10791 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:23:51 dist-test-slave-2w3w krb5kdc[25204](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:23:51 dist-test-slave-2w3w krb5kdc[25204](info): set up 2 sockets
Aug 07 11:23:51 dist-test-slave-2w3w krb5kdc[25204](info): commencing operation
krb5kdc: starting...
W20260807 11:23:52.194407 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:23:52 dist-test-slave-2w3w krb5kdc[25204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:23:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:23:52Z Disabled control of system clock
WARNING: no policy specified for oryx/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.62.keytab.
Entry for principal oryx/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.62.keytab.
I20260807 11:23:52.443353 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:42387
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.62.keytab
--principal=oryx/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:41227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:42387
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:52.777240 25217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:52.777590 25217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:52.777766 25217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:52.784999 25217 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:23:52.785157 25217 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 11:23:52.785262 25217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:52.785382 25217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:52.785477 25217 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:23:52.785566 25217 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:23:52.795645 25217 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:41227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:42387
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.62.keytab
--principal=oryx/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:42387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:52.797057 25217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:52.798480 25217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:52.811818 25222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:52.811829 25223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:52.812098 25225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:52.812687 25217 server_base.cc:1034] running on GCE node
I20260807 11:23:52.813529 25217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:52.815104 25217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:52.816383 25217 hybrid_clock.cc:648] HybridClock initialized: now 1786101832816298 us; error 56 us; skew 500 ppm
I20260807 11:23:52.816738 25217 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:52 dist-test-slave-2w3w krb5kdc[25204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101832, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:52.820031 25217 init.cc:377] Logged in from keytab as oryx/127.16.197.62@KRBTEST.COM (short username oryx)
I20260807 11:23:52.821565 25217 webserver.cc:460] Webserver started at http://127.16.197.62:44833/ using document root <none> and password file <none>
I20260807 11:23:52.822325 25217 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:52.822476 25217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:52.822824 25217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:52.825958 25217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "7065983468d844f2800fb30776891978"
format_stamp: "Formatted at 2026-08-07 11:23:52 on dist-test-slave-2w3w"
server_key: "efffc33aec746a318eb2ea2e4ff0a15b"
server_key_iv: "f0ad1062f885cacab2c4f099de934008"
server_key_version: "encryptionkey@0"
I20260807 11:23:52.826802 25217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "7065983468d844f2800fb30776891978"
format_stamp: "Formatted at 2026-08-07 11:23:52 on dist-test-slave-2w3w"
server_key: "efffc33aec746a318eb2ea2e4ff0a15b"
server_key_iv: "f0ad1062f885cacab2c4f099de934008"
server_key_version: "encryptionkey@0"
I20260807 11:23:52.832070 25217 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 11:23:52.835570 25232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:52.836987 25217 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 11:23:52.837139 25217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "7065983468d844f2800fb30776891978"
format_stamp: "Formatted at 2026-08-07 11:23:52 on dist-test-slave-2w3w"
server_key: "efffc33aec746a318eb2ea2e4ff0a15b"
server_key_iv: "f0ad1062f885cacab2c4f099de934008"
server_key_version: "encryptionkey@0"
I20260807 11:23:52.837324 25217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:52.850469 25217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:52.854504 25217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:52.854764 25217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:52.876544 25217 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:42387
I20260807 11:23:52.876550 25298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:42387 every 8 connection(s)
I20260807 11:23:52.878005 25217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:23:52.881882 25299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:52.887562 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 25217
I20260807 11:23:52.887792 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:23:52.888633 17172 external_mini_cluster.cc:1428] Setting key c5d5e910c65e401ba498c00465da8b71
I20260807 11:23:52.895502 25299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7065983468d844f2800fb30776891978: Bootstrap starting.
I20260807 11:23:52.899375 25299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7065983468d844f2800fb30776891978: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:52.900911 25299 log.cc:824] T 00000000000000000000000000000000 P 7065983468d844f2800fb30776891978: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:52.904680 25299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7065983468d844f2800fb30776891978: No bootstrap required, opened a new log
Aug 07 11:23:52 dist-test-slave-2w3w krb5kdc[25204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM
I20260807 11:23:52.914848 25299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7065983468d844f2800fb30776891978 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7065983468d844f2800fb30776891978" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42387 } }
I20260807 11:23:52.915413 25299 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7065983468d844f2800fb30776891978 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:52.915535 25299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7065983468d844f2800fb30776891978 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7065983468d844f2800fb30776891978, State: Initialized, Role: FOLLOWER
I20260807 11:23:52.916355 25299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7065983468d844f2800fb30776891978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7065983468d844f2800fb30776891978" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42387 } }
I20260807 11:23:52.916692 25299 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7065983468d844f2800fb30776891978 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:23:52.916852 25299 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7065983468d844f2800fb30776891978 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:23:52.917095 25299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7065983468d844f2800fb30776891978 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:52.920432 25299 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7065983468d844f2800fb30776891978 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7065983468d844f2800fb30776891978" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42387 } }
I20260807 11:23:52.921077 25299 leader_election.cc:302] T 00000000000000000000000000000000 P 7065983468d844f2800fb30776891978 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7065983468d844f2800fb30776891978; no voters:
I20260807 11:23:52.921926 25299 leader_election.cc:288] T 00000000000000000000000000000000 P 7065983468d844f2800fb30776891978 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:23:52.922104 25304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7065983468d844f2800fb30776891978 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:52.924731 25299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7065983468d844f2800fb30776891978 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:23:52.925931 25302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:52.891025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51048 (local address 127.16.197.62:42387)
0807 11:23:52.900072 (+ 9047us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:52.900090 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:52.900144 (+ 54us) server_negotiation.cc:407] Connection header received
0807 11:23:52.902232 (+ 2088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:52.902263 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:52.902693 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:52.903117 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:52.905091 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:52.906626 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:52.908612 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:52.909494 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:52.909692 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:52.914692 (+ 5000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:52.914701 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:52.914733 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:52.914781 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:52.919172 (+ 4391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:52.919977 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:52.919987 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:52.919996 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:52.920236 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:52.920686 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:52.920691 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:52.920694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:52.921515 (+ 821us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:52.921960 (+ 445us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:52.923080 (+ 1120us) server_negotiation.cc:299] Negotiation successful
0807 11:23:52.923808 (+ 728us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8716,"thread_start_us":8833,"threads_started":1}
I20260807 11:23:52.927130 25304 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7065983468d844f2800fb30776891978 [term 1 LEADER]: Becoming Leader. State: Replica: 7065983468d844f2800fb30776891978, State: Running, Role: LEADER
I20260807 11:23:52.927884 25304 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7065983468d844f2800fb30776891978 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7065983468d844f2800fb30776891978" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42387 } }
I20260807 11:23:52.931281 25306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7065983468d844f2800fb30776891978 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7065983468d844f2800fb30776891978. Latest consensus state: current_term: 1 leader_uuid: "7065983468d844f2800fb30776891978" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7065983468d844f2800fb30776891978" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42387 } } }
I20260807 11:23:52.931113 25305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7065983468d844f2800fb30776891978 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7065983468d844f2800fb30776891978" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7065983468d844f2800fb30776891978" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42387 } } }
I20260807 11:23:52.931667 25306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7065983468d844f2800fb30776891978 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:52.931667 25305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7065983468d844f2800fb30776891978 [sys.catalog]: This master's current role is: LEADER
I20260807 11:23:52.932760 25311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:23:52.938382 25311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:23:52.949302 25311 catalog_manager.cc:1347] Generated new cluster ID: 597eedc6c3844b9dafc7ab3afec6623b
I20260807 11:23:52.949476 25311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:23:52.964947 25311 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:23:52.967007 25311 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:23:52.979043 25311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7065983468d844f2800fb30776891978: Generated new TSK 0
I20260807 11:23:52.980278 25311 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.16.197.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.197.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.197.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.1.keytab.
Entry for principal oryx/127.16.197.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.1.keytab.
I20260807 11:23:53.055558 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:0
--local_ip_for_outbound_sockets=127.16.197.1
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:42387
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.1.keytab
--principal=oryx/127.16.197.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:41227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:53.409066 25326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:53.409360 25326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:53.409461 25326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:53.417110 25326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:53.417227 25326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:53.417412 25326 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 11:23:53.417538 25326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:23:53.427986 25326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:41227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.1.keytab
--principal=oryx/127.16.197.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.197.62:42387
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:53.429351 25326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:53.430774 25326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:53.444847 25331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:53.444763 25332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:53.444720 25334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:53.446061 25326 server_base.cc:1034] running on GCE node
I20260807 11:23:53.446779 25326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:53.447826 25326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:53.449074 25326 hybrid_clock.cc:648] HybridClock initialized: now 1786101833448996 us; error 56 us; skew 500 ppm
I20260807 11:23:53.449383 25326 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:53 dist-test-slave-2w3w krb5kdc[25204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101833, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:53.452847 25326 init.cc:377] Logged in from keytab as oryx/127.16.197.1@KRBTEST.COM (short username oryx)
I20260807 11:23:53.454360 25326 webserver.cc:460] Webserver started at http://127.16.197.1:32853/ using document root <none> and password file <none>
I20260807 11:23:53.455173 25326 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:53.455309 25326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:53.455644 25326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:53.458554 25326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-0/data/instance:
uuid: "7fea1e3cfe29487f979dc0929a53cef7"
format_stamp: "Formatted at 2026-08-07 11:23:53 on dist-test-slave-2w3w"
server_key: "bcb73dd389081da06eaf7ab7415659d0"
server_key_iv: "405f3d618741610364af18a159c8b6ff"
server_key_version: "encryptionkey@0"
I20260807 11:23:53.459427 25326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance:
uuid: "7fea1e3cfe29487f979dc0929a53cef7"
format_stamp: "Formatted at 2026-08-07 11:23:53 on dist-test-slave-2w3w"
server_key: "bcb73dd389081da06eaf7ab7415659d0"
server_key_iv: "405f3d618741610364af18a159c8b6ff"
server_key_version: "encryptionkey@0"
I20260807 11:23:53.464680 25326 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 11:23:53.468186 25341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:53.469556 25326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 11:23:53.469730 25326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "7fea1e3cfe29487f979dc0929a53cef7"
format_stamp: "Formatted at 2026-08-07 11:23:53 on dist-test-slave-2w3w"
server_key: "bcb73dd389081da06eaf7ab7415659d0"
server_key_iv: "405f3d618741610364af18a159c8b6ff"
server_key_version: "encryptionkey@0"
I20260807 11:23:53.469988 25326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:53.485613 25326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:53.490236 25326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:53.490545 25326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:53.493465 25326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:53.493582 25326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:53.493752 25326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:53.493834 25326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 11:23:53 dist-test-slave-2w3w krb5kdc[25204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101833, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM
I20260807 11:23:53.520969 25352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:53.494268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:37217 (local address 127.16.197.62:42387)
0807 11:23:53.494847 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:53.494853 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:53.496194 (+ 1341us) server_negotiation.cc:407] Connection header received
0807 11:23:53.498429 (+ 2235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:53.498435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:53.498535 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:53.498671 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:53.501069 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:53.502401 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:53.505183 (+ 2782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:53.506005 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:53.506157 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:53.512539 (+ 6382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:53.512545 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:53.512549 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:53.512586 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:53.516217 (+ 3631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:53.517481 (+ 1264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:53.517490 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:53.517495 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:53.517668 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:53.518248 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:53.518256 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:53.518262 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:53.518737 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:53.518912 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:53.520058 (+ 1146us) server_negotiation.cc:299] Negotiation successful
0807 11:23:53.520392 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":295,"threads_started":1}
I20260807 11:23:53.521627 25353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:53.495039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42387 (local address 127.16.197.1:37217)
0807 11:23:53.495996 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:53.496052 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:53.498104 (+ 2052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:53.498952 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:53.498991 (+ 39us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:53.499915 (+ 924us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:53.500821 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:53.500856 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:53.502631 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:53.502638 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:53.504962 (+ 2324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:53.504983 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:53.506580 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:53.506589 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:53.507002 (+ 413us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:53.508009 (+ 1007us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:53.508051 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:53.512169 (+ 4118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:53.516408 (+ 4239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:53.516422 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:53.516457 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:53.517320 (+ 863us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:53.517785 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:53.517791 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:53.517795 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:53.518094 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:53.518885 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:53.518891 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:53.519471 (+ 580us) client_negotiation.cc:769] Sending connection context
0807 11:23:53.519863 (+ 392us) client_negotiation.cc:240] Negotiation successful
0807 11:23:53.520358 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":406,"threads_started":1}
I20260807 11:23:53.539292 25469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:35809 every 8 connection(s)
I20260807 11:23:53.539327 25326 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:35809
I20260807 11:23:53.540947 25326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:23:53.543339 25470 heartbeater.cc:347] Connected to a master server at 127.16.197.62:42387
I20260807 11:23:53.543612 25470 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:53.544556 25470 heartbeater.cc:511] Master 127.16.197.62:42387 requested a full tablet report, sending...
I20260807 11:23:53.545643 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 25326
I20260807 11:23:53.545879 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance
I20260807 11:23:53.546068 25442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:53.529092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42387 (local address 127.16.197.1:48335)
0807 11:23:53.529840 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:53.529865 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:53.530066 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:53.530429 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:53.530433 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:53.531014 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:53.531640 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:53.531653 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:53.533002 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:53.533007 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:53.534802 (+ 1795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:53.534824 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:53.536378 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:53.536387 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:53.536656 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:53.537462 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:53.537485 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:53.538857 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:53.542788 (+ 3931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:53.542796 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:53.542800 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:53.543418 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:53.543923 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:53.543931 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:53.543934 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:53.544107 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:53.544830 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:53.544835 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:53.545059 (+ 224us) client_negotiation.cc:769] Sending connection context
0807 11:23:53.545206 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 11:23:53.545564 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":428,"threads_started":1}
I20260807 11:23:53.546793 17172 external_mini_cluster.cc:1428] Setting key 969d17f9a322378a4485509d6b7c73fa
I20260807 11:23:53.547091 25352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:53.529627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:48335 (local address 127.16.197.62:42387)
0807 11:23:53.529812 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:53.529819 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:53.529962 (+ 143us) server_negotiation.cc:407] Connection header received
0807 11:23:53.530210 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:53.530216 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:53.530290 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:53.530394 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:53.531794 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:53.532856 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:53.535065 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:53.535906 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:53.536063 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:53.539204 (+ 3141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:53.539213 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:53.539218 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:53.539259 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:53.542632 (+ 3373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:53.543602 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:53.543608 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:53.543612 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:53.543778 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:53.544271 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:53.544276 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:53.544278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:53.544672 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:53.546301 (+ 1629us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:53.546456 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 11:23:53.546691 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 11:23:53.547117 25250 ts_manager.cc:194] Registered new tserver with Master: 7fea1e3cfe29487f979dc0929a53cef7 (127.16.197.1:35809)
I20260807 11:23:53.549469 25250 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.197.1@KRBTEST.COM'} at 127.16.197.1:37217
WARNING: no policy specified for oryx/127.16.197.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.197.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.197.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.2.keytab.
Entry for principal oryx/127.16.197.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.2.keytab.
I20260807 11:23:53.602877 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:0
--local_ip_for_outbound_sockets=127.16.197.2
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:42387
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.2.keytab
--principal=oryx/127.16.197.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:41227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:53.944290 25475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:53.944588 25475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:53.944773 25475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:53.952092 25475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:53.952208 25475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:53.952394 25475 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 11:23:53.952518 25475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:23:53.962844 25475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:41227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.2.keytab
--principal=oryx/127.16.197.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.197.62:42387
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:53.964301 25475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:53.965787 25475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:53.979396 25481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:53.979357 25484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:53.979782 25482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:53.980688 25475 server_base.cc:1034] running on GCE node
I20260807 11:23:53.981339 25475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:53.982308 25475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:53.983532 25475 hybrid_clock.cc:648] HybridClock initialized: now 1786101833983458 us; error 43 us; skew 500 ppm
I20260807 11:23:53.983860 25475 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:53 dist-test-slave-2w3w krb5kdc[25204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101833, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:53.987123 25475 init.cc:377] Logged in from keytab as oryx/127.16.197.2@KRBTEST.COM (short username oryx)
I20260807 11:23:53.988658 25475 webserver.cc:460] Webserver started at http://127.16.197.2:37329/ using document root <none> and password file <none>
I20260807 11:23:53.989336 25475 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:53.989423 25475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:53.989702 25475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:53.992699 25475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-1/data/instance:
uuid: "b410615d35614d9db0b7ace7f1a19e8e"
format_stamp: "Formatted at 2026-08-07 11:23:53 on dist-test-slave-2w3w"
server_key: "122a76f5bc7acd3162482603b6874328"
server_key_iv: "b959b45b2f8f4f6cce06a8316416f5a4"
server_key_version: "encryptionkey@0"
I20260807 11:23:53.993477 25475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance:
uuid: "b410615d35614d9db0b7ace7f1a19e8e"
format_stamp: "Formatted at 2026-08-07 11:23:53 on dist-test-slave-2w3w"
server_key: "122a76f5bc7acd3162482603b6874328"
server_key_iv: "b959b45b2f8f4f6cce06a8316416f5a4"
server_key_version: "encryptionkey@0"
I20260807 11:23:53.998807 25475 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 11:23:54.005609 25491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:54.007501 25475 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.007s sys 0.000s
I20260807 11:23:54.007717 25475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "b410615d35614d9db0b7ace7f1a19e8e"
format_stamp: "Formatted at 2026-08-07 11:23:53 on dist-test-slave-2w3w"
server_key: "122a76f5bc7acd3162482603b6874328"
server_key_iv: "b959b45b2f8f4f6cce06a8316416f5a4"
server_key_version: "encryptionkey@0"
I20260807 11:23:54.008144 25475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:54.039208 25475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:54.043979 25475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:54.044270 25475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:54.047210 25475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:54.047326 25475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:54.047542 25475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:54.047658 25475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 11:23:54 dist-test-slave-2w3w krb5kdc[25204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101833, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM
I20260807 11:23:54.075543 25500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:54.048017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:51213 (local address 127.16.197.62:42387)
0807 11:23:54.051084 (+ 3067us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:54.051089 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:54.051109 (+ 20us) server_negotiation.cc:407] Connection header received
0807 11:23:54.053655 (+ 2546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:54.053664 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:54.053782 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:54.053926 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:54.056698 (+ 2772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:54.057933 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:54.060742 (+ 2809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:54.061414 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:54.061587 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:54.068271 (+ 6684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:54.068277 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:54.068280 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:54.068328 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:54.071294 (+ 2966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:54.072519 (+ 1225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:54.072529 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:54.072533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:54.072731 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:54.073343 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:54.073350 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:54.073355 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:54.073727 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:54.073971 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:54.074837 (+ 866us) server_negotiation.cc:299] Negotiation successful
0807 11:23:54.075129 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2919,"thread_start_us":2788,"threads_started":1}
I20260807 11:23:54.076697 25501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:54.048866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42387 (local address 127.16.197.2:51213)
0807 11:23:54.050007 (+ 1141us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:54.050075 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:54.053236 (+ 3161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:54.054402 (+ 1166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:54.054417 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:54.055378 (+ 961us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:54.056445 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:54.056482 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:54.058197 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:54.058205 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:54.060540 (+ 2335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:54.060570 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:54.061953 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:54.061959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:54.062321 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:54.063239 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:54.063283 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:54.067927 (+ 4644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:54.071547 (+ 3620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:54.071562 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:54.071590 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:54.072333 (+ 743us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:54.072855 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:54.072862 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:54.072868 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:54.073194 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:54.073935 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:54.073943 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:54.074353 (+ 410us) client_negotiation.cc:769] Sending connection context
0807 11:23:54.074712 (+ 359us) client_negotiation.cc:240] Negotiation successful
0807 11:23:54.075400 (+ 688us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":768,"thread_start_us":478,"threads_started":1}
I20260807 11:23:54.106034 25500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:54.087207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:43143 (local address 127.16.197.62:42387)
0807 11:23:54.087424 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:54.087429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:54.088389 (+ 960us) server_negotiation.cc:407] Connection header received
0807 11:23:54.088620 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:54.088628 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:54.088701 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:54.088810 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:54.090491 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:54.091556 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:54.094428 (+ 2872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:54.095273 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:54.095444 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:54.098404 (+ 2960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:54.098413 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:54.098421 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:54.098476 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:54.102140 (+ 3664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:54.103405 (+ 1265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:54.103414 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:54.103418 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:54.103616 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:54.104075 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:54.104079 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:54.104081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:54.104425 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:54.104601 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:54.105316 (+ 715us) server_negotiation.cc:299] Negotiation successful
0807 11:23:54.105602 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 11:23:54.105997 25475 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:36483
I20260807 11:23:54.106330 25573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:54.087569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42387 (local address 127.16.197.2:43143)
0807 11:23:54.088258 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:54.088276 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:54.088477 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:54.088853 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:54.088858 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:54.089728 (+ 870us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:54.090248 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:54.090271 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:54.091768 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:54.091772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:54.094226 (+ 2454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:54.094248 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:54.095503 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:54.095509 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:54.095694 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:54.096439 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:54.096468 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:54.098117 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:54.102350 (+ 4233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:54.102360 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:54.102367 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:54.103213 (+ 846us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:54.103784 (+ 571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:54.103788 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:54.103791 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:54.103961 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:54.104587 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:54.104600 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:54.104869 (+ 269us) client_negotiation.cc:769] Sending connection context
0807 11:23:54.105136 (+ 267us) client_negotiation.cc:240] Negotiation successful
0807 11:23:54.105695 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":404,"threads_started":1}
I20260807 11:23:54.106038 25621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:36483 every 8 connection(s)
I20260807 11:23:54.107793 25475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:23:54.108645 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 25475
I20260807 11:23:54.108826 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance
I20260807 11:23:54.109378 17172 external_mini_cluster.cc:1428] Setting key 38005cdf9650e71b48620c299cad6902
I20260807 11:23:54.111071 25622 heartbeater.cc:347] Connected to a master server at 127.16.197.62:42387
I20260807 11:23:54.111565 25622 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:54.112533 25622 heartbeater.cc:511] Master 127.16.197.62:42387 requested a full tablet report, sending...
I20260807 11:23:54.115350 25250 ts_manager.cc:194] Registered new tserver with Master: b410615d35614d9db0b7ace7f1a19e8e (127.16.197.2:36483)
I20260807 11:23:54.116158 25250 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.197.2@KRBTEST.COM'} at 127.16.197.2:51213
WARNING: no policy specified for oryx/127.16.197.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.197.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.197.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.3.keytab.
Entry for principal oryx/127.16.197.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.3.keytab.
I20260807 11:23:54.170686 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:0
--local_ip_for_outbound_sockets=127.16.197.3
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:42387
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.3.keytab
--principal=oryx/127.16.197.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:41227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:54.520056 25626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:54.520363 25626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:54.520467 25626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:54.528213 25626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:54.528343 25626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:54.528429 25626 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 11:23:54.528512 25626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:23:54.538849 25626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:41227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.3.keytab
--principal=oryx/127.16.197.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.197.62:42387
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:54.540563 25626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:54.542331 25626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 11:23:54.552491 25470 heartbeater.cc:503] Master 127.16.197.62:42387 was elected leader, sending a full tablet report...
W20260807 11:23:54.557087 25634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:54.557160 25631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:54.557188 25632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:54.558025 25626 server_base.cc:1034] running on GCE node
I20260807 11:23:54.558655 25626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:54.559710 25626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:54.560981 25626 hybrid_clock.cc:648] HybridClock initialized: now 1786101834560913 us; error 37 us; skew 500 ppm
I20260807 11:23:54.561334 25626 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:23:54 dist-test-slave-2w3w krb5kdc[25204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101834, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:23:54.564779 25626 init.cc:377] Logged in from keytab as oryx/127.16.197.3@KRBTEST.COM (short username oryx)
I20260807 11:23:54.566314 25626 webserver.cc:460] Webserver started at http://127.16.197.3:41373/ using document root <none> and password file <none>
I20260807 11:23:54.567121 25626 fs_manager.cc:359] Metadata directory not provided
I20260807 11:23:54.567260 25626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:23:54.567632 25626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:23:54.570628 25626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-2/data/instance:
uuid: "e5cb1ba24cf843f792962faf2870dbe7"
format_stamp: "Formatted at 2026-08-07 11:23:54 on dist-test-slave-2w3w"
server_key: "5fa6a2f93ac50e2f94d1ff16bd58eee0"
server_key_iv: "2d8a12023d89e53b242161f17fb0f7db"
server_key_version: "encryptionkey@0"
I20260807 11:23:54.571529 25626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance:
uuid: "e5cb1ba24cf843f792962faf2870dbe7"
format_stamp: "Formatted at 2026-08-07 11:23:54 on dist-test-slave-2w3w"
server_key: "5fa6a2f93ac50e2f94d1ff16bd58eee0"
server_key_iv: "2d8a12023d89e53b242161f17fb0f7db"
server_key_version: "encryptionkey@0"
I20260807 11:23:54.577168 25626 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 11:23:54.580822 25642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:54.582175 25626 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 11:23:54.582353 25626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "e5cb1ba24cf843f792962faf2870dbe7"
format_stamp: "Formatted at 2026-08-07 11:23:54 on dist-test-slave-2w3w"
server_key: "5fa6a2f93ac50e2f94d1ff16bd58eee0"
server_key_iv: "2d8a12023d89e53b242161f17fb0f7db"
server_key_version: "encryptionkey@0"
I20260807 11:23:54.582617 25626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:23:54.603936 25626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:23:54.608815 25626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:23:54.609195 25626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:23:54.612190 25626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:23:54.612304 25626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:23:54.612483 25626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:23:54.612632 25626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 07 11:23:54 dist-test-slave-2w3w krb5kdc[25204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101834, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM
I20260807 11:23:54.641321 25650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:54.612693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:47593 (local address 127.16.197.62:42387)
0807 11:23:54.613253 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:54.613260 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:54.614902 (+ 1642us) server_negotiation.cc:407] Connection header received
0807 11:23:54.618099 (+ 3197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:54.618107 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:54.618201 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:54.618332 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:54.620900 (+ 2568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:54.622001 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:54.624837 (+ 2836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:54.625607 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:54.625769 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:54.631752 (+ 5983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:54.631758 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:54.631763 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:54.631803 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:54.636183 (+ 4380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:54.637644 (+ 1461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:54.637652 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:54.637659 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:54.637852 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:54.638652 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:54.638657 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:54.638661 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:54.639194 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:54.639436 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:54.640308 (+ 872us) server_negotiation.cc:299] Negotiation successful
0807 11:23:54.640660 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":268,"threads_started":1}
I20260807 11:23:54.642030 25653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:54.613664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42387 (local address 127.16.197.3:47593)
0807 11:23:54.614698 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:54.614777 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:54.617702 (+ 2925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:54.618673 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:54.618687 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:54.619683 (+ 996us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:54.620693 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:54.620724 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:54.622180 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:54.622193 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:54.624670 (+ 2477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:54.624688 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:54.625898 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:54.625913 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:54.626398 (+ 485us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:54.627429 (+ 1031us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:54.627481 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:54.631350 (+ 3869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:54.636439 (+ 5089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:54.636453 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:54.636475 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:54.637397 (+ 922us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:54.638035 (+ 638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:54.638042 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:54.638046 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:54.638476 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:54.639377 (+ 901us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:54.639384 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:54.639780 (+ 396us) client_negotiation.cc:769] Sending connection context
0807 11:23:54.640065 (+ 285us) client_negotiation.cc:240] Negotiation successful
0807 11:23:54.640501 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":393,"threads_started":1}
I20260807 11:23:54.665942 25650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:54.651670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:46289 (local address 127.16.197.62:42387)
0807 11:23:54.651871 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:54.651877 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:54.652703 (+ 826us) server_negotiation.cc:407] Connection header received
0807 11:23:54.652908 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:54.652912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:54.652989 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:54.653110 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:54.654893 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:54.656012 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:54.657427 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:54.657862 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:54.658008 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:54.660410 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:54.660417 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:54.660422 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:54.660463 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:54.663250 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:54.663927 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:54.663933 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:54.663937 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:54.664069 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:54.664442 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:54.664448 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:54.664451 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:54.664730 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:54.664920 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:54.665309 (+ 389us) server_negotiation.cc:299] Negotiation successful
0807 11:23:54.665551 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 11:23:54.666057 25744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:54.651851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42387 (local address 127.16.197.3:46289)
0807 11:23:54.652570 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:54.652594 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:54.652762 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:54.653145 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:54.653150 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:54.654010 (+ 860us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:54.654677 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:54.654691 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:54.656197 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:54.656203 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:54.657269 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:54.657283 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:54.658056 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:54.658062 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:54.658229 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:54.658891 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:54.658909 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:54.660208 (+ 1299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:54.663385 (+ 3177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:54.663391 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:54.663395 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:54.663803 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:54.664184 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:54.664189 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:54.664193 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:54.664321 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:54.664856 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:54.664862 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:54.665054 (+ 192us) client_negotiation.cc:769] Sending connection context
0807 11:23:54.665199 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 11:23:54.665548 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":347,"threads_started":1}
I20260807 11:23:54.671931 25626 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:46859
I20260807 11:23:54.672020 25771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:46859 every 8 connection(s)
I20260807 11:23:54.673272 25626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:23:54.675666 25772 heartbeater.cc:347] Connected to a master server at 127.16.197.62:42387
I20260807 11:23:54.675940 25772 heartbeater.cc:464] Registering TS with master...
I20260807 11:23:54.676651 25772 heartbeater.cc:511] Master 127.16.197.62:42387 requested a full tablet report, sending...
I20260807 11:23:54.676873 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 25626
I20260807 11:23:54.677052 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance
I20260807 11:23:54.677558 17172 external_mini_cluster.cc:1428] Setting key 758c88d310ef2405befbd53c9772c4ca
I20260807 11:23:54.678318 25250 ts_manager.cc:194] Registered new tserver with Master: e5cb1ba24cf843f792962faf2870dbe7 (127.16.197.3:46859)
I20260807 11:23:54.679477 25250 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.197.3@KRBTEST.COM'} at 127.16.197.3:47593
I20260807 11:23:54.681085 17172 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 11:23:54 dist-test-slave-2w3w krb5kdc[25204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101834, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 11:23:54 dist-test-slave-2w3w krb5kdc[25204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:23:54 dist-test-slave-2w3w krb5kdc[25204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:23:55.119598 25622 heartbeater.cc:503] Master 127.16.197.62:42387 was elected leader, sending a full tablet report...
I20260807 11:23:55.682548 25772 heartbeater.cc:503] Master 127.16.197.62:42387 was elected leader, sending a full tablet report...
W20260807 11:23:55.894893 25780 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.161s user 0.000s sys 0.018s
W20260807 11:23:55.896279 25650 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:23:54.728161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51056 (local address 127.16.197.62:42387)
0807 11:23:54.728431 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:54.728437 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:54.728652 (+ 215us) server_negotiation.cc:407] Connection header received
0807 11:23:54.728842 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:54.728847 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:54.728920 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:54.729016 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:54.730496 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:54.731291 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:54.732600 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:54.733020 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:54.733129 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:55.895927 (+1162798us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51056: BlockingRecv error: recv got EOF from 127.0.0.1:51056 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 07 11:23:55 dist-test-slave-2w3w krb5kdc[25204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101834, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM
I20260807 11:23:55.920713 25650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:55.900514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51070 (local address 127.16.197.62:42387)
0807 11:23:55.900762 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:55.900770 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:55.901041 (+ 271us) server_negotiation.cc:407] Connection header received
0807 11:23:55.901268 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:55.901272 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:55.901339 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:55.901450 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:55.902741 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:55.903546 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:55.904840 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:55.905265 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:55.905405 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:55.915313 (+ 9908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:55.915319 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:55.915323 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:55.915362 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:55.917935 (+ 2573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:55.918621 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:55.918632 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:55.918635 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:55.918759 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:55.919228 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:55.919234 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:55.919237 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:55.919555 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:55.919857 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:55.920145 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 11:23:55.920390 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 11:23:55.924829 25250 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51070:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 11:23:55.932063 25250 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:23:55.967476 25794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:55.944596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:36483 (local address 127.0.0.1:41944)
0807 11:23:55.958471 (+ 13875us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:55.958531 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:55.958803 (+ 272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:55.959670 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:55.959681 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:55.959720 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:55.960259 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:55.960278 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:55.962279 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:55.962285 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:55.964788 (+ 2503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:55.964801 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:55.966380 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:55.966387 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:55.966592 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:55.966647 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 11:23:55.966784 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 11:23:55.967015 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13532,"mutex_wait_us":76,"thread_start_us":15901,"threads_started":1}
I20260807 11:23:55.968520 25793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:55.944772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41944 (local address 127.16.197.2:36483)
0807 11:23:55.957682 (+ 12910us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:55.957695 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:55.958869 (+ 1174us) server_negotiation.cc:407] Connection header received
0807 11:23:55.959085 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:55.959096 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:55.959413 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:55.959746 (+ 333us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:55.960850 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:55.962143 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:55.964965 (+ 2822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:55.966149 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:55.966304 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:55.967901 (+ 1597us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:55.967981 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 11:23:55.968214 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12647,"thread_start_us":14102,"threads_started":1}
I20260807 11:23:55.968571 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:55.946170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:35809 (local address 127.0.0.1:44048)
0807 11:23:55.960292 (+ 14122us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:55.960318 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:55.960444 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:55.961146 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:55.961152 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:55.961179 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:55.961522 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:55.961533 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:55.963462 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:55.963468 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:55.965721 (+ 2253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:55.965735 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:55.967931 (+ 2196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:55.967942 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:55.968127 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:55.968152 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 11:23:55.968216 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 11:23:55.968307 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13973,"thread_start_us":13066,"threads_started":1}
I20260807 11:23:55.969250 25795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:55.945238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53814 (local address 127.16.197.3:46859)
0807 11:23:55.952227 (+ 6989us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:55.952245 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:55.959668 (+ 7423us) server_negotiation.cc:407] Connection header received
0807 11:23:55.959882 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:55.959893 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:55.960157 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:55.960360 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:55.961240 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:55.962718 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:55.965761 (+ 3043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:55.967255 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:55.967392 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:55.967604 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:55.968777 (+ 1173us) server_negotiation.cc:299] Negotiation successful
0807 11:23:55.968961 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":326,"threads_started":1}
I20260807 11:23:55.969319 25797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:55.946421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44048 (local address 127.16.197.1:35809)
0807 11:23:55.958702 (+ 12281us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:55.958710 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:55.960577 (+ 1867us) server_negotiation.cc:407] Connection header received
0807 11:23:55.960680 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:55.960690 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:55.960921 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:55.961137 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:55.961705 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:55.963262 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:55.965854 (+ 2592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:55.967291 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:55.967434 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:55.968815 (+ 1381us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:55.968878 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 11:23:55.969034 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12109,"thread_start_us":12281,"threads_started":1}
I20260807 11:23:55.968647 25796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:55.944785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:46859 (local address 127.0.0.1:53814)
0807 11:23:55.959541 (+ 14756us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:55.959569 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:55.959709 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:55.960610 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:55.960614 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:55.960639 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:55.961047 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:55.961064 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:55.962898 (+ 1834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:55.962914 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:55.965575 (+ 2661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:55.965592 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:55.967770 (+ 2178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:55.967776 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:55.967994 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:55.968024 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 11:23:55.968113 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 11:23:55.968251 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14601,"mutex_wait_us":84,"thread_start_us":13195,"threads_started":1}
I20260807 11:23:55.972693 25547 tablet_service.cc:1467] Processing CreateTablet for tablet f8370ea434f4472daade12259b4eeb14 (DEFAULT_TABLE table=test-table [id=6a6d8b89b7a2425fb0f4980421ebee78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:55.972862 25396 tablet_service.cc:1467] Processing CreateTablet for tablet f8370ea434f4472daade12259b4eeb14 (DEFAULT_TABLE table=test-table [id=6a6d8b89b7a2425fb0f4980421ebee78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:55.972864 25697 tablet_service.cc:1467] Processing CreateTablet for tablet f8370ea434f4472daade12259b4eeb14 (DEFAULT_TABLE table=test-table [id=6a6d8b89b7a2425fb0f4980421ebee78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:55.974146 25547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8370ea434f4472daade12259b4eeb14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:55.974256 25697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8370ea434f4472daade12259b4eeb14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:55.974251 25396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8370ea434f4472daade12259b4eeb14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:55.986136 25798 tablet_bootstrap.cc:492] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e: Bootstrap starting.
I20260807 11:23:55.989434 25798 tablet_bootstrap.cc:654] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:55.990797 25798 log.cc:824] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:55.993582 25798 tablet_bootstrap.cc:492] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e: No bootstrap required, opened a new log
I20260807 11:23:55.993769 25800 tablet_bootstrap.cc:492] T f8370ea434f4472daade12259b4eeb14 P 7fea1e3cfe29487f979dc0929a53cef7: Bootstrap starting.
I20260807 11:23:55.993899 25798 ts_tablet_manager.cc:1397] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260807 11:23:55.995680 25801 tablet_bootstrap.cc:492] T f8370ea434f4472daade12259b4eeb14 P e5cb1ba24cf843f792962faf2870dbe7: Bootstrap starting.
I20260807 11:23:55.997618 25800 tablet_bootstrap.cc:654] T f8370ea434f4472daade12259b4eeb14 P 7fea1e3cfe29487f979dc0929a53cef7: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:55.999024 25800 log.cc:824] T f8370ea434f4472daade12259b4eeb14 P 7fea1e3cfe29487f979dc0929a53cef7: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:55.999562 25801 tablet_bootstrap.cc:654] T f8370ea434f4472daade12259b4eeb14 P e5cb1ba24cf843f792962faf2870dbe7: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:56.000857 25801 log.cc:824] T f8370ea434f4472daade12259b4eeb14 P e5cb1ba24cf843f792962faf2870dbe7: Log is configured to *not* fsync() on all Append() calls
I20260807 11:23:56.002390 25800 tablet_bootstrap.cc:492] T f8370ea434f4472daade12259b4eeb14 P 7fea1e3cfe29487f979dc0929a53cef7: No bootstrap required, opened a new log
I20260807 11:23:56.002684 25800 ts_tablet_manager.cc:1397] T f8370ea434f4472daade12259b4eeb14 P 7fea1e3cfe29487f979dc0929a53cef7: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260807 11:23:56.003669 25801 tablet_bootstrap.cc:492] T f8370ea434f4472daade12259b4eeb14 P e5cb1ba24cf843f792962faf2870dbe7: No bootstrap required, opened a new log
I20260807 11:23:56.003945 25801 ts_tablet_manager.cc:1397] T f8370ea434f4472daade12259b4eeb14 P e5cb1ba24cf843f792962faf2870dbe7: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260807 11:23:56.004595 25798 raft_consensus.cc:348] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } } peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } }
I20260807 11:23:56.005050 25798 raft_consensus.cc:374] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:56.005203 25798 raft_consensus.cc:729] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b410615d35614d9db0b7ace7f1a19e8e, State: Initialized, Role: FOLLOWER
I20260807 11:23:56.005872 25798 consensus_queue.cc:260] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } } peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } }
I20260807 11:23:56.007707 25798 ts_tablet_manager.cc:1428] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260807 11:23:56.012955 25800 raft_consensus.cc:348] T f8370ea434f4472daade12259b4eeb14 P 7fea1e3cfe29487f979dc0929a53cef7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } } peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } }
I20260807 11:23:56.013376 25800 raft_consensus.cc:374] T f8370ea434f4472daade12259b4eeb14 P 7fea1e3cfe29487f979dc0929a53cef7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:56.013510 25800 raft_consensus.cc:729] T f8370ea434f4472daade12259b4eeb14 P 7fea1e3cfe29487f979dc0929a53cef7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fea1e3cfe29487f979dc0929a53cef7, State: Initialized, Role: FOLLOWER
I20260807 11:23:56.013792 25801 raft_consensus.cc:348] T f8370ea434f4472daade12259b4eeb14 P e5cb1ba24cf843f792962faf2870dbe7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } } peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } }
I20260807 11:23:56.014259 25801 raft_consensus.cc:374] T f8370ea434f4472daade12259b4eeb14 P e5cb1ba24cf843f792962faf2870dbe7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:56.014377 25801 raft_consensus.cc:729] T f8370ea434f4472daade12259b4eeb14 P e5cb1ba24cf843f792962faf2870dbe7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5cb1ba24cf843f792962faf2870dbe7, State: Initialized, Role: FOLLOWER
I20260807 11:23:56.014245 25800 consensus_queue.cc:260] T f8370ea434f4472daade12259b4eeb14 P 7fea1e3cfe29487f979dc0929a53cef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } } peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } }
I20260807 11:23:56.015070 25801 consensus_queue.cc:260] T f8370ea434f4472daade12259b4eeb14 P e5cb1ba24cf843f792962faf2870dbe7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } } peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } }
I20260807 11:23:56.017446 25800 ts_tablet_manager.cc:1428] T f8370ea434f4472daade12259b4eeb14 P 7fea1e3cfe29487f979dc0929a53cef7: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260807 11:23:56.019282 25801 ts_tablet_manager.cc:1428] T f8370ea434f4472daade12259b4eeb14 P e5cb1ba24cf843f792962faf2870dbe7: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
W20260807 11:23:56.050007 25472 tablet.cc:2369] T f8370ea434f4472daade12259b4eeb14 P 7fea1e3cfe29487f979dc0929a53cef7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:23:56.055537 25806 raft_consensus.cc:482] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:23:56.055877 25806 raft_consensus.cc:504] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } } peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } }
I20260807 11:23:56.057614 25806 leader_election.cc:288] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5cb1ba24cf843f792962faf2870dbe7 (127.16.197.3:46859), 7fea1e3cfe29487f979dc0929a53cef7 (127.16.197.1:35809)
I20260807 11:23:56.067920 25797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:56.058229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:39661 (local address 127.16.197.1:35809)
0807 11:23:56.058500 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:56.058511 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:56.059914 (+ 1403us) server_negotiation.cc:407] Connection header received
0807 11:23:56.060073 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:56.060078 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:56.060189 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:56.060280 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:56.061123 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.062238 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.065298 (+ 3060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.066534 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.066695 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:56.066772 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:56.067475 (+ 703us) server_negotiation.cc:299] Negotiation successful
0807 11:23:56.067612 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 11:23:56.067962 25795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:56.058704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:47207 (local address 127.16.197.3:46859)
0807 11:23:56.058945 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:56.058953 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:56.059733 (+ 780us) server_negotiation.cc:407] Connection header received
0807 11:23:56.059953 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:56.059959 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:56.060095 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:56.060214 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:56.061123 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.062238 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.065288 (+ 3050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.066534 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.066695 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:56.066772 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:56.067475 (+ 703us) server_negotiation.cc:299] Negotiation successful
0807 11:23:56.067612 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 11:23:56.068188 25809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:56.058074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:35809 (local address 127.16.197.2:39661)
0807 11:23:56.059801 (+ 1727us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:56.059819 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:56.059943 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:56.060295 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:56.060299 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:56.060315 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:56.060963 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:56.060978 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.062377 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.062381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:56.065115 (+ 2734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:56.065131 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.066836 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.066841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:56.067176 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:56.067331 (+ 155us) client_negotiation.cc:769] Sending connection context
0807 11:23:56.067434 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 11:23:56.067646 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1604,"mutex_wait_us":299,"spinlock_wait_cycles":4992,"thread_start_us":1098,"threads_started":1}
I20260807 11:23:56.068292 25810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:56.057915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:46859 (local address 127.16.197.2:47207)
0807 11:23:56.059584 (+ 1669us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:56.059608 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:56.059769 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:56.060266 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:56.060272 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:56.060300 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:56.060963 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:56.060978 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.062375 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.062380 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:56.065119 (+ 2739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:56.065131 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.066829 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.066836 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:56.067172 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:56.067331 (+ 159us) client_negotiation.cc:769] Sending connection context
0807 11:23:56.067442 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 11:23:56.067646 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1494,"mutex_wait_us":414,"thread_start_us":1604,"threads_started":1}
I20260807 11:23:56.069198 25417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8370ea434f4472daade12259b4eeb14" candidate_uuid: "b410615d35614d9db0b7ace7f1a19e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fea1e3cfe29487f979dc0929a53cef7" is_pre_election: true
I20260807 11:23:56.069312 25718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8370ea434f4472daade12259b4eeb14" candidate_uuid: "b410615d35614d9db0b7ace7f1a19e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5cb1ba24cf843f792962faf2870dbe7" is_pre_election: true
I20260807 11:23:56.069746 25417 raft_consensus.cc:2454] T f8370ea434f4472daade12259b4eeb14 P 7fea1e3cfe29487f979dc0929a53cef7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b410615d35614d9db0b7ace7f1a19e8e in term 0.
I20260807 11:23:56.069821 25718 raft_consensus.cc:2454] T f8370ea434f4472daade12259b4eeb14 P e5cb1ba24cf843f792962faf2870dbe7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b410615d35614d9db0b7ace7f1a19e8e in term 0.
I20260807 11:23:56.070484 25495 leader_election.cc:302] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7fea1e3cfe29487f979dc0929a53cef7, b410615d35614d9db0b7ace7f1a19e8e, e5cb1ba24cf843f792962faf2870dbe7; no voters:
I20260807 11:23:56.070904 25806 raft_consensus.cc:2781] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:23:56.071087 25806 raft_consensus.cc:482] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:23:56.071208 25806 raft_consensus.cc:3037] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:56.073472 25806 raft_consensus.cc:504] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } } peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } }
I20260807 11:23:56.074105 25806 leader_election.cc:288] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e [CANDIDATE]: Term 1 election: Requested vote from peers e5cb1ba24cf843f792962faf2870dbe7 (127.16.197.3:46859), 7fea1e3cfe29487f979dc0929a53cef7 (127.16.197.1:35809)
I20260807 11:23:56.074642 25718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8370ea434f4472daade12259b4eeb14" candidate_uuid: "b410615d35614d9db0b7ace7f1a19e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5cb1ba24cf843f792962faf2870dbe7"
I20260807 11:23:56.074640 25417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8370ea434f4472daade12259b4eeb14" candidate_uuid: "b410615d35614d9db0b7ace7f1a19e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fea1e3cfe29487f979dc0929a53cef7"
I20260807 11:23:56.074882 25417 raft_consensus.cc:3037] T f8370ea434f4472daade12259b4eeb14 P 7fea1e3cfe29487f979dc0929a53cef7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:56.074882 25718 raft_consensus.cc:3037] T f8370ea434f4472daade12259b4eeb14 P e5cb1ba24cf843f792962faf2870dbe7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:56.078233 25718 raft_consensus.cc:2454] T f8370ea434f4472daade12259b4eeb14 P e5cb1ba24cf843f792962faf2870dbe7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b410615d35614d9db0b7ace7f1a19e8e in term 1.
I20260807 11:23:56.078238 25417 raft_consensus.cc:2454] T f8370ea434f4472daade12259b4eeb14 P 7fea1e3cfe29487f979dc0929a53cef7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b410615d35614d9db0b7ace7f1a19e8e in term 1.
I20260807 11:23:56.078724 25495 leader_election.cc:302] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fea1e3cfe29487f979dc0929a53cef7, b410615d35614d9db0b7ace7f1a19e8e; no voters:
I20260807 11:23:56.079087 25806 raft_consensus.cc:2781] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:56.079612 25806 raft_consensus.cc:686] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e [term 1 LEADER]: Becoming Leader. State: Replica: b410615d35614d9db0b7ace7f1a19e8e, State: Running, Role: LEADER
I20260807 11:23:56.080154 25806 consensus_queue.cc:237] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } } peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } }
I20260807 11:23:56.084836 25250 catalog_manager.cc:5495] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e reported cstate change: term changed from 0 to 1, leader changed from <none> to b410615d35614d9db0b7ace7f1a19e8e (127.16.197.2). New cstate: current_term: 1 leader_uuid: "b410615d35614d9db0b7ace7f1a19e8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } health_report { overall_health: UNKNOWN } } }
W20260807 11:23:56.116014 25623 tablet.cc:2369] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:23:56.126818 25793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:56.121164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41956 (local address 127.16.197.2:36483)
0807 11:23:56.121363 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:56.121370 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:56.121479 (+ 109us) server_negotiation.cc:407] Connection header received
0807 11:23:56.121773 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:56.121777 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:56.121883 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:56.122053 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:23:56.122427 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.123230 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.124716 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.125109 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.125203 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:56.125663 (+ 460us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:23:56.126220 (+ 557us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:23:56.126328 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:56.126428 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 11:23:56.126501 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 11:23:56.135583 25718 raft_consensus.cc:1261] T f8370ea434f4472daade12259b4eeb14 P e5cb1ba24cf843f792962faf2870dbe7 [term 1 FOLLOWER]: Refusing update from remote peer b410615d35614d9db0b7ace7f1a19e8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:56.135583 25417 raft_consensus.cc:1261] T f8370ea434f4472daade12259b4eeb14 P 7fea1e3cfe29487f979dc0929a53cef7 [term 1 FOLLOWER]: Refusing update from remote peer b410615d35614d9db0b7ace7f1a19e8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:56.136673 25811 consensus_queue.cc:1035] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:56.137045 25806 consensus_queue.cc:1035] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:56.151422 25815 mvcc.cc:205] Tried to move back new op lower bound from 7315873120801280000 to 7315873120587964416. Current Snapshot: MvccSnapshot[applied={T|T < 7315873120801280000}]
I20260807 11:23:56.167029 25250 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:23:56.167316 25250 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:23:56.171228 25250 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7065983468d844f2800fb30776891978: Sending DeleteTablet for 3 replicas of tablet f8370ea434f4472daade12259b4eeb14
I20260807 11:23:56.172528 25547 tablet_service.cc:1514] Processing DeleteTablet for tablet f8370ea434f4472daade12259b4eeb14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:56 UTC) from {username='oryx', principal='oryx/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:41944
I20260807 11:23:56.172519 25697 tablet_service.cc:1514] Processing DeleteTablet for tablet f8370ea434f4472daade12259b4eeb14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:56 UTC) from {username='oryx', principal='oryx/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:53814
I20260807 11:23:56.172940 25396 tablet_service.cc:1514] Processing DeleteTablet for tablet f8370ea434f4472daade12259b4eeb14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:56 UTC) from {username='oryx', principal='oryx/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:44048
I20260807 11:23:56.174779 25250 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51070:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 11:23:56.177749 25250 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:23:56.190676 25546 tablet_service.cc:1467] Processing CreateTablet for tablet 61c22d0038984394907ee276a06b3b46 (DEFAULT_TABLE table=test-table [id=10abddb313754cd3bf9999c6376fc40c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:56.190927 25696 tablet_service.cc:1467] Processing CreateTablet for tablet 61c22d0038984394907ee276a06b3b46 (DEFAULT_TABLE table=test-table [id=10abddb313754cd3bf9999c6376fc40c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:56.191429 25546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61c22d0038984394907ee276a06b3b46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:56.191524 25696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61c22d0038984394907ee276a06b3b46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:56.191700 25395 tablet_service.cc:1467] Processing CreateTablet for tablet 61c22d0038984394907ee276a06b3b46 (DEFAULT_TABLE table=test-table [id=10abddb313754cd3bf9999c6376fc40c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:56.192461 25395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61c22d0038984394907ee276a06b3b46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:56.194310 25826 tablet_replica.cc:330] T f8370ea434f4472daade12259b4eeb14 P 7fea1e3cfe29487f979dc0929a53cef7: stopping tablet replica
I20260807 11:23:56.194841 25826 raft_consensus.cc:2229] T f8370ea434f4472daade12259b4eeb14 P 7fea1e3cfe29487f979dc0929a53cef7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:56.195420 25826 raft_consensus.cc:2258] T f8370ea434f4472daade12259b4eeb14 P 7fea1e3cfe29487f979dc0929a53cef7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:56.197597 25826 ts_tablet_manager.cc:1905] T f8370ea434f4472daade12259b4eeb14 P 7fea1e3cfe29487f979dc0929a53cef7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:56.198725 25798 tablet_bootstrap.cc:492] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e: Bootstrap starting.
I20260807 11:23:56.200160 25800 tablet_bootstrap.cc:492] T 61c22d0038984394907ee276a06b3b46 P 7fea1e3cfe29487f979dc0929a53cef7: Bootstrap starting.
I20260807 11:23:56.200812 25798 tablet_bootstrap.cc:654] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:56.201900 25800 tablet_bootstrap.cc:654] T 61c22d0038984394907ee276a06b3b46 P 7fea1e3cfe29487f979dc0929a53cef7: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:56.205164 25826 ts_tablet_manager.cc:1918] T f8370ea434f4472daade12259b4eeb14 P 7fea1e3cfe29487f979dc0929a53cef7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:23:56.205458 25826 log.cc:1196] T f8370ea434f4472daade12259b4eeb14 P 7fea1e3cfe29487f979dc0929a53cef7: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-0/wal/wals/f8370ea434f4472daade12259b4eeb14
I20260807 11:23:56.206143 25826 ts_tablet_manager.cc:1939] T f8370ea434f4472daade12259b4eeb14 P 7fea1e3cfe29487f979dc0929a53cef7: Deleting consensus metadata
I20260807 11:23:56.208241 25236 catalog_manager.cc:4841] TS 7fea1e3cfe29487f979dc0929a53cef7 (127.16.197.1:35809): tablet f8370ea434f4472daade12259b4eeb14 (table test-table [id=6a6d8b89b7a2425fb0f4980421ebee78]) successfully deleted
I20260807 11:23:56.210278 25801 tablet_bootstrap.cc:492] T 61c22d0038984394907ee276a06b3b46 P e5cb1ba24cf843f792962faf2870dbe7: Bootstrap starting.
I20260807 11:23:56.212098 25801 tablet_bootstrap.cc:654] T 61c22d0038984394907ee276a06b3b46 P e5cb1ba24cf843f792962faf2870dbe7: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:56.214653 25798 tablet_bootstrap.cc:492] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e: No bootstrap required, opened a new log
I20260807 11:23:56.214855 25798 ts_tablet_manager.cc:1397] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.003s
I20260807 11:23:56.216066 25798 raft_consensus.cc:348] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:56.216540 25798 raft_consensus.cc:374] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:56.216717 25798 raft_consensus.cc:729] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b410615d35614d9db0b7ace7f1a19e8e, State: Initialized, Role: FOLLOWER
I20260807 11:23:56.217187 25798 consensus_queue.cc:260] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:56.218281 25798 ts_tablet_manager.cc:1428] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 11:23:56.218883 25825 tablet_replica.cc:330] T f8370ea434f4472daade12259b4eeb14 P e5cb1ba24cf843f792962faf2870dbe7: stopping tablet replica
I20260807 11:23:56.223951 25800 tablet_bootstrap.cc:492] T 61c22d0038984394907ee276a06b3b46 P 7fea1e3cfe29487f979dc0929a53cef7: No bootstrap required, opened a new log
I20260807 11:23:56.224155 25800 ts_tablet_manager.cc:1397] T 61c22d0038984394907ee276a06b3b46 P 7fea1e3cfe29487f979dc0929a53cef7: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.007s
I20260807 11:23:56.226260 25800 raft_consensus.cc:348] T 61c22d0038984394907ee276a06b3b46 P 7fea1e3cfe29487f979dc0929a53cef7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:56.226778 25800 raft_consensus.cc:374] T 61c22d0038984394907ee276a06b3b46 P 7fea1e3cfe29487f979dc0929a53cef7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:56.226914 25800 raft_consensus.cc:729] T 61c22d0038984394907ee276a06b3b46 P 7fea1e3cfe29487f979dc0929a53cef7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fea1e3cfe29487f979dc0929a53cef7, State: Initialized, Role: FOLLOWER
I20260807 11:23:56.225677 25801 tablet_bootstrap.cc:492] T 61c22d0038984394907ee276a06b3b46 P e5cb1ba24cf843f792962faf2870dbe7: No bootstrap required, opened a new log
I20260807 11:23:56.227306 25801 ts_tablet_manager.cc:1397] T 61c22d0038984394907ee276a06b3b46 P e5cb1ba24cf843f792962faf2870dbe7: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.003s
I20260807 11:23:56.227720 25800 consensus_queue.cc:260] T 61c22d0038984394907ee276a06b3b46 P 7fea1e3cfe29487f979dc0929a53cef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:56.228179 25801 raft_consensus.cc:348] T 61c22d0038984394907ee276a06b3b46 P e5cb1ba24cf843f792962faf2870dbe7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:56.228412 25801 raft_consensus.cc:374] T 61c22d0038984394907ee276a06b3b46 P e5cb1ba24cf843f792962faf2870dbe7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:56.228514 25801 raft_consensus.cc:729] T 61c22d0038984394907ee276a06b3b46 P e5cb1ba24cf843f792962faf2870dbe7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5cb1ba24cf843f792962faf2870dbe7, State: Initialized, Role: FOLLOWER
I20260807 11:23:56.228806 25801 consensus_queue.cc:260] T 61c22d0038984394907ee276a06b3b46 P e5cb1ba24cf843f792962faf2870dbe7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:56.229383 25801 ts_tablet_manager.cc:1428] T 61c22d0038984394907ee276a06b3b46 P e5cb1ba24cf843f792962faf2870dbe7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 11:23:56.229938 25825 raft_consensus.cc:2229] T f8370ea434f4472daade12259b4eeb14 P e5cb1ba24cf843f792962faf2870dbe7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:56.230423 25825 raft_consensus.cc:2258] T f8370ea434f4472daade12259b4eeb14 P e5cb1ba24cf843f792962faf2870dbe7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:56.231252 25824 tablet_replica.cc:330] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e: stopping tablet replica
I20260807 11:23:56.231326 25800 ts_tablet_manager.cc:1428] T 61c22d0038984394907ee276a06b3b46 P 7fea1e3cfe29487f979dc0929a53cef7: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260807 11:23:56.231768 25824 raft_consensus.cc:2229] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e [term 1 LEADER]: Raft consensus shutting down.
I20260807 11:23:56.232597 25824 raft_consensus.cc:2258] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:56.232686 25825 ts_tablet_manager.cc:1905] T f8370ea434f4472daade12259b4eeb14 P e5cb1ba24cf843f792962faf2870dbe7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:56.237578 25824 ts_tablet_manager.cc:1905] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:56.239921 25825 ts_tablet_manager.cc:1918] T f8370ea434f4472daade12259b4eeb14 P e5cb1ba24cf843f792962faf2870dbe7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:23:56.240103 25825 log.cc:1196] T f8370ea434f4472daade12259b4eeb14 P e5cb1ba24cf843f792962faf2870dbe7: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-2/wal/wals/f8370ea434f4472daade12259b4eeb14
I20260807 11:23:56.240702 25825 ts_tablet_manager.cc:1939] T f8370ea434f4472daade12259b4eeb14 P e5cb1ba24cf843f792962faf2870dbe7: Deleting consensus metadata
I20260807 11:23:56.242676 25235 catalog_manager.cc:4841] TS e5cb1ba24cf843f792962faf2870dbe7 (127.16.197.3:46859): tablet f8370ea434f4472daade12259b4eeb14 (table test-table [id=6a6d8b89b7a2425fb0f4980421ebee78]) successfully deleted
I20260807 11:23:56.245198 25824 ts_tablet_manager.cc:1918] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:23:56.245358 25824 log.cc:1196] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-1/wal/wals/f8370ea434f4472daade12259b4eeb14
I20260807 11:23:56.245777 25824 ts_tablet_manager.cc:1939] T f8370ea434f4472daade12259b4eeb14 P b410615d35614d9db0b7ace7f1a19e8e: Deleting consensus metadata
I20260807 11:23:56.247249 25233 catalog_manager.cc:4841] TS b410615d35614d9db0b7ace7f1a19e8e (127.16.197.2:36483): tablet f8370ea434f4472daade12259b4eeb14 (table test-table [id=6a6d8b89b7a2425fb0f4980421ebee78]) successfully deleted
I20260807 11:23:56.270432 25811 raft_consensus.cc:482] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:23:56.270705 25811 raft_consensus.cc:504] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:56.271574 25811 leader_election.cc:288] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7fea1e3cfe29487f979dc0929a53cef7 (127.16.197.1:35809), e5cb1ba24cf843f792962faf2870dbe7 (127.16.197.3:46859)
I20260807 11:23:56.272233 25718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61c22d0038984394907ee276a06b3b46" candidate_uuid: "b410615d35614d9db0b7ace7f1a19e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5cb1ba24cf843f792962faf2870dbe7" is_pre_election: true
I20260807 11:23:56.272250 25417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61c22d0038984394907ee276a06b3b46" candidate_uuid: "b410615d35614d9db0b7ace7f1a19e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fea1e3cfe29487f979dc0929a53cef7" is_pre_election: true
I20260807 11:23:56.272506 25718 raft_consensus.cc:2454] T 61c22d0038984394907ee276a06b3b46 P e5cb1ba24cf843f792962faf2870dbe7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b410615d35614d9db0b7ace7f1a19e8e in term 0.
I20260807 11:23:56.272523 25417 raft_consensus.cc:2454] T 61c22d0038984394907ee276a06b3b46 P 7fea1e3cfe29487f979dc0929a53cef7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b410615d35614d9db0b7ace7f1a19e8e in term 0.
I20260807 11:23:56.273013 25495 leader_election.cc:302] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fea1e3cfe29487f979dc0929a53cef7, b410615d35614d9db0b7ace7f1a19e8e; no voters:
I20260807 11:23:56.273368 25811 raft_consensus.cc:2781] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:23:56.273519 25811 raft_consensus.cc:482] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:23:56.273653 25811 raft_consensus.cc:3037] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:56.276139 25811 raft_consensus.cc:504] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:56.276754 25811 leader_election.cc:288] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e [CANDIDATE]: Term 1 election: Requested vote from peers 7fea1e3cfe29487f979dc0929a53cef7 (127.16.197.1:35809), e5cb1ba24cf843f792962faf2870dbe7 (127.16.197.3:46859)
I20260807 11:23:56.277308 25417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61c22d0038984394907ee276a06b3b46" candidate_uuid: "b410615d35614d9db0b7ace7f1a19e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fea1e3cfe29487f979dc0929a53cef7"
I20260807 11:23:56.277315 25718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61c22d0038984394907ee276a06b3b46" candidate_uuid: "b410615d35614d9db0b7ace7f1a19e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5cb1ba24cf843f792962faf2870dbe7"
I20260807 11:23:56.277534 25417 raft_consensus.cc:3037] T 61c22d0038984394907ee276a06b3b46 P 7fea1e3cfe29487f979dc0929a53cef7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:56.277547 25718 raft_consensus.cc:3037] T 61c22d0038984394907ee276a06b3b46 P e5cb1ba24cf843f792962faf2870dbe7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:56.279893 25417 raft_consensus.cc:2454] T 61c22d0038984394907ee276a06b3b46 P 7fea1e3cfe29487f979dc0929a53cef7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b410615d35614d9db0b7ace7f1a19e8e in term 1.
I20260807 11:23:56.280367 25495 leader_election.cc:302] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fea1e3cfe29487f979dc0929a53cef7, b410615d35614d9db0b7ace7f1a19e8e; no voters:
I20260807 11:23:56.280648 25811 raft_consensus.cc:2781] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:56.280659 25718 raft_consensus.cc:2454] T 61c22d0038984394907ee276a06b3b46 P e5cb1ba24cf843f792962faf2870dbe7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b410615d35614d9db0b7ace7f1a19e8e in term 1.
I20260807 11:23:56.280879 25811 raft_consensus.cc:686] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e [term 1 LEADER]: Becoming Leader. State: Replica: b410615d35614d9db0b7ace7f1a19e8e, State: Running, Role: LEADER
I20260807 11:23:56.281147 25811 consensus_queue.cc:237] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:56.284471 25248 catalog_manager.cc:5495] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e reported cstate change: term changed from 0 to 1, leader changed from <none> to b410615d35614d9db0b7ace7f1a19e8e (127.16.197.2). New cstate: current_term: 1 leader_uuid: "b410615d35614d9db0b7ace7f1a19e8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } health_report { overall_health: UNKNOWN } } }
Aug 07 11:23:56 dist-test-slave-2w3w krb5kdc[25204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101832, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.62@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM
I20260807 11:23:56.353139 25650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:56.337774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51082 (local address 127.16.197.62:42387)
0807 11:23:56.338009 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:56.338014 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:56.338627 (+ 613us) server_negotiation.cc:407] Connection header received
0807 11:23:56.338855 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:56.338860 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:56.338933 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:56.339082 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:56.340323 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.341188 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.342524 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.342939 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.343122 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:56.347335 (+ 4213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:56.347340 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:56.347344 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:56.347375 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:56.350167 (+ 2792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:56.350918 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:56.350926 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:56.350934 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:56.351119 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:56.351527 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:56.351533 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:56.351537 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:56.351865 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:56.352026 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:56.352476 (+ 450us) server_negotiation.cc:299] Negotiation successful
0807 11:23:56.352723 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 11:23:56.353308 25837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:56.337845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42387 (local address 127.0.0.1:51082)
0807 11:23:56.338509 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:56.338526 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:56.338714 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:56.339148 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:56.339153 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:56.339894 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:23:56.340168 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:56.340178 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.341327 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.341335 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:56.342374 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:56.342386 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.343164 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.343171 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:56.343333 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:56.344121 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:23:56.344156 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:23:56.347087 (+ 2931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:23:56.350325 (+ 3238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:56.350340 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:56.350348 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:56.350774 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:56.351229 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:23:56.351234 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:23:56.351238 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:23:56.351391 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:23:56.352017 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:23:56.352025 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:23:56.352205 (+ 180us) client_negotiation.cc:769] Sending connection context
0807 11:23:56.352376 (+ 171us) client_negotiation.cc:240] Negotiation successful
0807 11:23:56.352876 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"mutex_wait_us":174,"thread_start_us":396,"threads_started":1}
I20260807 11:23:56.358831 25248 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:51082:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 11:23:56.360049 25248 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:23:56.367297 25395 tablet_service.cc:1467] Processing CreateTablet for tablet 1c0d7594c55345d0afa0612a50cfab71 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=13e3e5d4761e49548dcd1bfb8793bfb5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 11:23:56.367357 25547 tablet_service.cc:1467] Processing CreateTablet for tablet 1c0d7594c55345d0afa0612a50cfab71 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=13e3e5d4761e49548dcd1bfb8793bfb5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 11:23:56.367664 25697 tablet_service.cc:1467] Processing CreateTablet for tablet 1c0d7594c55345d0afa0612a50cfab71 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=13e3e5d4761e49548dcd1bfb8793bfb5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 11:23:56.367946 25395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c0d7594c55345d0afa0612a50cfab71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:56.367964 25547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c0d7594c55345d0afa0612a50cfab71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:56.368275 25697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c0d7594c55345d0afa0612a50cfab71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:56.375628 25798 tablet_bootstrap.cc:492] T 1c0d7594c55345d0afa0612a50cfab71 P b410615d35614d9db0b7ace7f1a19e8e: Bootstrap starting.
I20260807 11:23:56.375639 25800 tablet_bootstrap.cc:492] T 1c0d7594c55345d0afa0612a50cfab71 P 7fea1e3cfe29487f979dc0929a53cef7: Bootstrap starting.
I20260807 11:23:56.376106 25801 tablet_bootstrap.cc:492] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7: Bootstrap starting.
I20260807 11:23:56.377492 25800 tablet_bootstrap.cc:654] T 1c0d7594c55345d0afa0612a50cfab71 P 7fea1e3cfe29487f979dc0929a53cef7: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:56.377507 25798 tablet_bootstrap.cc:654] T 1c0d7594c55345d0afa0612a50cfab71 P b410615d35614d9db0b7ace7f1a19e8e: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:56.377851 25801 tablet_bootstrap.cc:654] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:56.387562 25800 tablet_bootstrap.cc:492] T 1c0d7594c55345d0afa0612a50cfab71 P 7fea1e3cfe29487f979dc0929a53cef7: No bootstrap required, opened a new log
I20260807 11:23:56.387810 25800 ts_tablet_manager.cc:1397] T 1c0d7594c55345d0afa0612a50cfab71 P 7fea1e3cfe29487f979dc0929a53cef7: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260807 11:23:56.388881 25800 raft_consensus.cc:348] T 1c0d7594c55345d0afa0612a50cfab71 P 7fea1e3cfe29487f979dc0929a53cef7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:56.389216 25800 raft_consensus.cc:374] T 1c0d7594c55345d0afa0612a50cfab71 P 7fea1e3cfe29487f979dc0929a53cef7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:56.389302 25800 raft_consensus.cc:729] T 1c0d7594c55345d0afa0612a50cfab71 P 7fea1e3cfe29487f979dc0929a53cef7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fea1e3cfe29487f979dc0929a53cef7, State: Initialized, Role: FOLLOWER
I20260807 11:23:56.389613 25800 consensus_queue.cc:260] T 1c0d7594c55345d0afa0612a50cfab71 P 7fea1e3cfe29487f979dc0929a53cef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:56.390143 25807 tablet_replica.cc:439] T 1c0d7594c55345d0afa0612a50cfab71 P 7fea1e3cfe29487f979dc0929a53cef7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } } }
I20260807 11:23:56.390516 25807 tablet_replica.cc:442] T 1c0d7594c55345d0afa0612a50cfab71 P 7fea1e3cfe29487f979dc0929a53cef7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 11:23:56.390792 25798 tablet_bootstrap.cc:492] T 1c0d7594c55345d0afa0612a50cfab71 P b410615d35614d9db0b7ace7f1a19e8e: No bootstrap required, opened a new log
I20260807 11:23:56.391012 25798 ts_tablet_manager.cc:1397] T 1c0d7594c55345d0afa0612a50cfab71 P b410615d35614d9db0b7ace7f1a19e8e: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.011s
I20260807 11:23:56.391585 25800 ts_tablet_manager.cc:1428] T 1c0d7594c55345d0afa0612a50cfab71 P 7fea1e3cfe29487f979dc0929a53cef7: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260807 11:23:56.391884 25798 raft_consensus.cc:348] T 1c0d7594c55345d0afa0612a50cfab71 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:56.392172 25798 raft_consensus.cc:374] T 1c0d7594c55345d0afa0612a50cfab71 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:56.392266 25798 raft_consensus.cc:729] T 1c0d7594c55345d0afa0612a50cfab71 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b410615d35614d9db0b7ace7f1a19e8e, State: Initialized, Role: FOLLOWER
I20260807 11:23:56.392555 25798 consensus_queue.cc:260] T 1c0d7594c55345d0afa0612a50cfab71 P b410615d35614d9db0b7ace7f1a19e8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:56.393067 25811 tablet_replica.cc:439] T 1c0d7594c55345d0afa0612a50cfab71 P b410615d35614d9db0b7ace7f1a19e8e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } } }
I20260807 11:23:56.393354 25811 tablet_replica.cc:442] T 1c0d7594c55345d0afa0612a50cfab71 P b410615d35614d9db0b7ace7f1a19e8e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 11:23:56.393932 25798 ts_tablet_manager.cc:1428] T 1c0d7594c55345d0afa0612a50cfab71 P b410615d35614d9db0b7ace7f1a19e8e: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260807 11:23:56.396703 25801 tablet_bootstrap.cc:492] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7: No bootstrap required, opened a new log
I20260807 11:23:56.396917 25801 ts_tablet_manager.cc:1397] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.006s
I20260807 11:23:56.399295 25801 raft_consensus.cc:348] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:56.399590 25801 raft_consensus.cc:374] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:56.399686 25801 raft_consensus.cc:729] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5cb1ba24cf843f792962faf2870dbe7, State: Initialized, Role: FOLLOWER
I20260807 11:23:56.399941 25801 consensus_queue.cc:260] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:56.400575 25801 ts_tablet_manager.cc:1428] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260807 11:23:56.400463 25808 tablet_replica.cc:439] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } } }
I20260807 11:23:56.400873 25808 tablet_replica.cc:442] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7: This TxnStatusTablet replica's current role is: FOLLOWER
W20260807 11:23:56.427645 25773 tablet.cc:2369] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:23:56.467530 25808 raft_consensus.cc:482] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:23:56.467779 25808 raft_consensus.cc:504] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:56.469419 25808 leader_election.cc:288] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7fea1e3cfe29487f979dc0929a53cef7 (127.16.197.1:35809), b410615d35614d9db0b7ace7f1a19e8e (127.16.197.2:36483)
I20260807 11:23:56.478696 25797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:56.470347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:55817 (local address 127.16.197.1:35809)
0807 11:23:56.470565 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:56.470572 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:56.472314 (+ 1742us) server_negotiation.cc:407] Connection header received
0807 11:23:56.472653 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:56.472659 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:56.472756 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:56.472872 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:56.473562 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.474740 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.476719 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.477628 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.477811 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:56.477902 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:56.478243 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 11:23:56.478390 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 11:23:56.478709 25844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:56.469950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:35809 (local address 127.16.197.3:55817)
0807 11:23:56.472174 (+ 2224us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:56.472202 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:56.472457 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:56.472904 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:56.472908 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:56.472935 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:56.473323 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:56.473339 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.474900 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.474913 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:56.476545 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:56.476559 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.477867 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.477875 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:56.478051 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:56.478091 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 11:23:56.478192 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 11:23:56.478331 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2055,"thread_start_us":2279,"threads_started":1}
I20260807 11:23:56.479382 25417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c0d7594c55345d0afa0612a50cfab71" candidate_uuid: "e5cb1ba24cf843f792962faf2870dbe7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fea1e3cfe29487f979dc0929a53cef7" is_pre_election: true
I20260807 11:23:56.479713 25417 raft_consensus.cc:2454] T 1c0d7594c55345d0afa0612a50cfab71 P 7fea1e3cfe29487f979dc0929a53cef7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5cb1ba24cf843f792962faf2870dbe7 in term 0.
I20260807 11:23:56.480242 25843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:56.470034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:36483 (local address 127.16.197.3:57347)
0807 11:23:56.472574 (+ 2540us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:56.472591 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:56.472739 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:56.473251 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:56.473259 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:56.473291 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:56.473705 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:56.473715 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.475059 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.475065 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:56.477288 (+ 2223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:56.477305 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.479419 (+ 2114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.479427 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:56.479610 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:56.479642 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 11:23:56.479746 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 11:23:56.479883 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2405,"thread_start_us":1782,"threads_started":1}
I20260807 11:23:56.480345 25646 leader_election.cc:302] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fea1e3cfe29487f979dc0929a53cef7, e5cb1ba24cf843f792962faf2870dbe7; no voters:
I20260807 11:23:56.480556 25793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:56.470304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:57347 (local address 127.16.197.2:36483)
0807 11:23:56.470591 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:56.470600 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:56.472694 (+ 2094us) server_negotiation.cc:407] Connection header received
0807 11:23:56.472884 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:56.472889 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:56.472983 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:56.473082 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:56.473901 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.474901 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.477476 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.478750 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.478935 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:56.479251 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:56.479797 (+ 546us) server_negotiation.cc:299] Negotiation successful
0807 11:23:56.480027 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 11:23:56.480993 25808 raft_consensus.cc:2781] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:23:56.481108 25808 raft_consensus.cc:482] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:23:56.481189 25808 raft_consensus.cc:3037] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:56.481567 25572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c0d7594c55345d0afa0612a50cfab71" candidate_uuid: "e5cb1ba24cf843f792962faf2870dbe7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b410615d35614d9db0b7ace7f1a19e8e" is_pre_election: true
I20260807 11:23:56.482052 25572 raft_consensus.cc:2454] T 1c0d7594c55345d0afa0612a50cfab71 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5cb1ba24cf843f792962faf2870dbe7 in term 0.
I20260807 11:23:56.483824 25808 raft_consensus.cc:504] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:56.484467 25808 leader_election.cc:288] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7 [CANDIDATE]: Term 1 election: Requested vote from peers 7fea1e3cfe29487f979dc0929a53cef7 (127.16.197.1:35809), b410615d35614d9db0b7ace7f1a19e8e (127.16.197.2:36483)
I20260807 11:23:56.485013 25417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c0d7594c55345d0afa0612a50cfab71" candidate_uuid: "e5cb1ba24cf843f792962faf2870dbe7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fea1e3cfe29487f979dc0929a53cef7"
I20260807 11:23:56.485244 25417 raft_consensus.cc:3037] T 1c0d7594c55345d0afa0612a50cfab71 P 7fea1e3cfe29487f979dc0929a53cef7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:56.487203 25572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c0d7594c55345d0afa0612a50cfab71" candidate_uuid: "e5cb1ba24cf843f792962faf2870dbe7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b410615d35614d9db0b7ace7f1a19e8e"
I20260807 11:23:56.487440 25572 raft_consensus.cc:3037] T 1c0d7594c55345d0afa0612a50cfab71 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:56.487648 25417 raft_consensus.cc:2454] T 1c0d7594c55345d0afa0612a50cfab71 P 7fea1e3cfe29487f979dc0929a53cef7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5cb1ba24cf843f792962faf2870dbe7 in term 1.
I20260807 11:23:56.488085 25646 leader_election.cc:302] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fea1e3cfe29487f979dc0929a53cef7, e5cb1ba24cf843f792962faf2870dbe7; no voters:
I20260807 11:23:56.488377 25808 raft_consensus.cc:2781] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:56.488934 25808 raft_consensus.cc:686] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7 [term 1 LEADER]: Becoming Leader. State: Replica: e5cb1ba24cf843f792962faf2870dbe7, State: Running, Role: LEADER
I20260807 11:23:56.489396 25808 consensus_queue.cc:237] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:56.492795 25572 raft_consensus.cc:2454] T 1c0d7594c55345d0afa0612a50cfab71 P b410615d35614d9db0b7ace7f1a19e8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5cb1ba24cf843f792962faf2870dbe7 in term 1.
I20260807 11:23:56.494513 25845 tablet_replica.cc:439] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7: TxnStatusTablet state changed. Reason: New leader e5cb1ba24cf843f792962faf2870dbe7. Latest consensus state: current_term: 1 leader_uuid: "e5cb1ba24cf843f792962faf2870dbe7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } } }
I20260807 11:23:56.494874 25845 tablet_replica.cc:442] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7: This TxnStatusTablet replica's current role is: LEADER
I20260807 11:23:56.495620 25847 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 11:23:56.495419 25248 catalog_manager.cc:5495] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7 reported cstate change: term changed from 0 to 1, leader changed from <none> to e5cb1ba24cf843f792962faf2870dbe7 (127.16.197.3). New cstate: current_term: 1 leader_uuid: "e5cb1ba24cf843f792962faf2870dbe7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } health_report { overall_health: HEALTHY } } }
I20260807 11:23:56.495852 25847 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 11:23:56.529119 25837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:56.522736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:46859 (local address 127.0.0.1:53820)
0807 11:23:56.522932 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:56.522948 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:56.523155 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:56.523716 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:56.523720 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:56.523732 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 11:23:56.523946 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:56.523956 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.525226 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.525232 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:56.526575 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:56.526591 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.527463 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.527469 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:56.527646 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:56.527702 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 11:23:56.528552 (+ 850us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 11:23:56.528578 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 11:23:56.528668 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 11:23:56.528822 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 11:23:56.529165 25795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:56.522595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53820 (local address 127.16.197.3:46859)
0807 11:23:56.522781 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:56.522789 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:56.523064 (+ 275us) server_negotiation.cc:407] Connection header received
0807 11:23:56.523394 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:56.523402 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:56.523519 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:56.523695 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:23:56.524087 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.525098 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.526768 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.527297 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.527443 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:56.527916 (+ 473us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:23:56.528437 (+ 521us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:23:56.528604 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:56.528685 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 11:23:56.528775 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 11:23:56.533599 25417 raft_consensus.cc:1261] T 1c0d7594c55345d0afa0612a50cfab71 P 7fea1e3cfe29487f979dc0929a53cef7 [term 1 FOLLOWER]: Refusing update from remote peer e5cb1ba24cf843f792962faf2870dbe7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:56.533692 25572 raft_consensus.cc:1261] T 1c0d7594c55345d0afa0612a50cfab71 P b410615d35614d9db0b7ace7f1a19e8e [term 1 FOLLOWER]: Refusing update from remote peer e5cb1ba24cf843f792962faf2870dbe7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:56.534566 25845 consensus_queue.cc:1035] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:56.534931 25808 consensus_queue.cc:1035] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:56.555008 25807 tablet_replica.cc:439] T 1c0d7594c55345d0afa0612a50cfab71 P 7fea1e3cfe29487f979dc0929a53cef7: TxnStatusTablet state changed. Reason: New leader e5cb1ba24cf843f792962faf2870dbe7. Latest consensus state: current_term: 1 leader_uuid: "e5cb1ba24cf843f792962faf2870dbe7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } } }
I20260807 11:23:56.555373 25807 tablet_replica.cc:442] T 1c0d7594c55345d0afa0612a50cfab71 P 7fea1e3cfe29487f979dc0929a53cef7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 11:23:56.560603 25845 tablet_replica.cc:439] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e5cb1ba24cf843f792962faf2870dbe7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } } }
I20260807 11:23:56.560998 25845 tablet_replica.cc:442] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7: This TxnStatusTablet replica's current role is: LEADER
I20260807 11:23:56.570026 25807 tablet_replica.cc:439] T 1c0d7594c55345d0afa0612a50cfab71 P 7fea1e3cfe29487f979dc0929a53cef7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e5cb1ba24cf843f792962faf2870dbe7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } } }
I20260807 11:23:56.570391 25807 tablet_replica.cc:442] T 1c0d7594c55345d0afa0612a50cfab71 P 7fea1e3cfe29487f979dc0929a53cef7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 11:23:56.586939 25811 tablet_replica.cc:439] T 1c0d7594c55345d0afa0612a50cfab71 P b410615d35614d9db0b7ace7f1a19e8e: TxnStatusTablet state changed. Reason: New leader e5cb1ba24cf843f792962faf2870dbe7. Latest consensus state: current_term: 1 leader_uuid: "e5cb1ba24cf843f792962faf2870dbe7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } } }
I20260807 11:23:56.587311 25811 tablet_replica.cc:442] T 1c0d7594c55345d0afa0612a50cfab71 P b410615d35614d9db0b7ace7f1a19e8e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 11:23:56.591336 25808 tablet_replica.cc:439] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e5cb1ba24cf843f792962faf2870dbe7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } } }
I20260807 11:23:56.591660 25808 tablet_replica.cc:442] T 1c0d7594c55345d0afa0612a50cfab71 P e5cb1ba24cf843f792962faf2870dbe7: This TxnStatusTablet replica's current role is: LEADER
I20260807 11:23:56.593237 25814 mvcc.cc:205] Tried to move back new op lower bound from 7315873122433327104 to 7315873122267021312. Current Snapshot: MvccSnapshot[applied={T|T < 7315873122433327104}]
I20260807 11:23:56.594074 25811 tablet_replica.cc:439] T 1c0d7594c55345d0afa0612a50cfab71 P b410615d35614d9db0b7ace7f1a19e8e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e5cb1ba24cf843f792962faf2870dbe7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } } }
I20260807 11:23:56.594373 25811 tablet_replica.cc:442] T 1c0d7594c55345d0afa0612a50cfab71 P b410615d35614d9db0b7ace7f1a19e8e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 11:23:56.613549 25795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:56.605989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:60807 (local address 127.16.197.3:46859)
0807 11:23:56.606238 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:56.606247 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:56.607706 (+ 1459us) server_negotiation.cc:407] Connection header received
0807 11:23:56.607961 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:56.607965 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:56.608057 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:56.608165 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:23:56.608682 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.609617 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.611471 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.611915 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.612094 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:56.612557 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:23:56.612676 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:23:56.612814 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:56.613005 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 11:23:56.613087 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 11:23:56.614107 25854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:56.605688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:46859 (local address 127.16.197.2:60807)
0807 11:23:56.607578 (+ 1890us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:56.607596 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:56.607790 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:56.608184 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:56.608188 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:56.608200 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 11:23:56.608512 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:56.608522 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.609771 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.609778 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:56.611305 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:56.611328 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.612120 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.612126 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:56.612301 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:56.612361 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 11:23:56.612859 (+ 498us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 11:23:56.612868 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 11:23:56.612978 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 11:23:56.613118 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1733,"thread_start_us":1519,"threads_started":1}
I20260807 11:23:56.626652 25718 raft_consensus.cc:1261] T 61c22d0038984394907ee276a06b3b46 P e5cb1ba24cf843f792962faf2870dbe7 [term 1 FOLLOWER]: Refusing update from remote peer b410615d35614d9db0b7ace7f1a19e8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:56.626859 25417 raft_consensus.cc:1261] T 61c22d0038984394907ee276a06b3b46 P 7fea1e3cfe29487f979dc0929a53cef7 [term 1 FOLLOWER]: Refusing update from remote peer b410615d35614d9db0b7ace7f1a19e8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:56.627569 25806 consensus_queue.cc:1035] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:56.627841 25811 consensus_queue.cc:1035] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:56.649817 25816 mvcc.cc:205] Tried to move back new op lower bound from 7315873122815844352 to 7315873121410265088. Current Snapshot: MvccSnapshot[applied={T|T < 7315873122815844352}]
I20260807 11:23:56.719077 25793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:56.698485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:57221 (local address 127.16.197.2:36483)
0807 11:23:56.698690 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:56.698696 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:56.711734 (+ 13038us) server_negotiation.cc:407] Connection header received
0807 11:23:56.711964 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:56.711970 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:56.712055 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:56.712132 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:23:56.713075 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.714454 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.716783 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.717482 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.717577 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:56.718221 (+ 644us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:23:56.718361 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:23:56.718434 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:56.718656 (+ 222us) server_negotiation.cc:299] Negotiation successful
0807 11:23:56.718744 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 11:23:56.719956 25859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:56.697849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:36483 (local address 127.16.197.3:57221)
0807 11:23:56.711610 (+ 13761us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:56.711628 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:56.711810 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:56.712333 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:56.712338 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:56.712351 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 11:23:56.712859 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:56.712873 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.714692 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.714697 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:56.716618 (+ 1921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:56.716633 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.717769 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.717776 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:56.717951 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:56.718022 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 11:23:56.718516 (+ 494us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 11:23:56.718526 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 11:23:56.719368 (+ 842us) client_negotiation.cc:240] Negotiation successful
0807 11:23:56.719532 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13530,"thread_start_us":13349,"threads_started":1}
I20260807 11:23:56.768599 25248 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:23:56.768862 25248 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:23:56.772066 25248 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7065983468d844f2800fb30776891978: Sending DeleteTablet for 3 replicas of tablet 61c22d0038984394907ee276a06b3b46
I20260807 11:23:56.772974 25395 tablet_service.cc:1514] Processing DeleteTablet for tablet 61c22d0038984394907ee276a06b3b46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:56 UTC) from {username='oryx', principal='oryx/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:44048
I20260807 11:23:56.773003 25547 tablet_service.cc:1514] Processing DeleteTablet for tablet 61c22d0038984394907ee276a06b3b46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:56 UTC) from {username='oryx', principal='oryx/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:41944
I20260807 11:23:56.779363 25697 tablet_service.cc:1514] Processing DeleteTablet for tablet 61c22d0038984394907ee276a06b3b46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:56 UTC) from {username='oryx', principal='oryx/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:53814
I20260807 11:23:56.783854 25863 tablet_replica.cc:330] T 61c22d0038984394907ee276a06b3b46 P e5cb1ba24cf843f792962faf2870dbe7: stopping tablet replica
I20260807 11:23:56.784674 25863 raft_consensus.cc:2229] T 61c22d0038984394907ee276a06b3b46 P e5cb1ba24cf843f792962faf2870dbe7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:56.785321 25863 raft_consensus.cc:2258] T 61c22d0038984394907ee276a06b3b46 P e5cb1ba24cf843f792962faf2870dbe7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:56.787602 25863 ts_tablet_manager.cc:1905] T 61c22d0038984394907ee276a06b3b46 P e5cb1ba24cf843f792962faf2870dbe7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:56.792896 25862 tablet_replica.cc:330] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e: stopping tablet replica
I20260807 11:23:56.793144 25861 tablet_replica.cc:330] T 61c22d0038984394907ee276a06b3b46 P 7fea1e3cfe29487f979dc0929a53cef7: stopping tablet replica
I20260807 11:23:56.793375 25861 raft_consensus.cc:2229] T 61c22d0038984394907ee276a06b3b46 P 7fea1e3cfe29487f979dc0929a53cef7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:56.793422 25862 raft_consensus.cc:2229] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e [term 1 LEADER]: Raft consensus shutting down.
I20260807 11:23:56.793623 25861 raft_consensus.cc:2258] T 61c22d0038984394907ee276a06b3b46 P 7fea1e3cfe29487f979dc0929a53cef7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:56.794833 25861 ts_tablet_manager.cc:1905] T 61c22d0038984394907ee276a06b3b46 P 7fea1e3cfe29487f979dc0929a53cef7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:56.795616 25862 raft_consensus.cc:2258] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:56.796803 25862 ts_tablet_manager.cc:1905] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:56.800393 25863 ts_tablet_manager.cc:1918] T 61c22d0038984394907ee276a06b3b46 P e5cb1ba24cf843f792962faf2870dbe7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 11:23:56.802207 25861 ts_tablet_manager.cc:1918] T 61c22d0038984394907ee276a06b3b46 P 7fea1e3cfe29487f979dc0929a53cef7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 11:23:56.802425 25861 log.cc:1196] T 61c22d0038984394907ee276a06b3b46 P 7fea1e3cfe29487f979dc0929a53cef7: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-0/wal/wals/61c22d0038984394907ee276a06b3b46
I20260807 11:23:56.802979 25861 ts_tablet_manager.cc:1939] T 61c22d0038984394907ee276a06b3b46 P 7fea1e3cfe29487f979dc0929a53cef7: Deleting consensus metadata
I20260807 11:23:56.804232 25236 catalog_manager.cc:4841] TS 7fea1e3cfe29487f979dc0929a53cef7 (127.16.197.1:35809): tablet 61c22d0038984394907ee276a06b3b46 (table test-table [id=10abddb313754cd3bf9999c6376fc40c]) successfully deleted
I20260807 11:23:56.807379 25862 ts_tablet_manager.cc:1918] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 11:23:56.807538 25862 log.cc:1196] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-1/wal/wals/61c22d0038984394907ee276a06b3b46
I20260807 11:23:56.807685 25863 log.cc:1196] T 61c22d0038984394907ee276a06b3b46 P e5cb1ba24cf843f792962faf2870dbe7: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-2/wal/wals/61c22d0038984394907ee276a06b3b46
I20260807 11:23:56.808140 25862 ts_tablet_manager.cc:1939] T 61c22d0038984394907ee276a06b3b46 P b410615d35614d9db0b7ace7f1a19e8e: Deleting consensus metadata
I20260807 11:23:56.808233 25863 ts_tablet_manager.cc:1939] T 61c22d0038984394907ee276a06b3b46 P e5cb1ba24cf843f792962faf2870dbe7: Deleting consensus metadata
I20260807 11:23:56.809514 25235 catalog_manager.cc:4841] TS e5cb1ba24cf843f792962faf2870dbe7 (127.16.197.3:46859): tablet 61c22d0038984394907ee276a06b3b46 (table test-table [id=10abddb313754cd3bf9999c6376fc40c]) successfully deleted
I20260807 11:23:56.809763 25233 catalog_manager.cc:4841] TS b410615d35614d9db0b7ace7f1a19e8e (127.16.197.2:36483): tablet 61c22d0038984394907ee276a06b3b46 (table test-table [id=10abddb313754cd3bf9999c6376fc40c]) successfully deleted
Aug 07 11:23:56 dist-test-slave-2w3w krb5kdc[25204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 11:23:56 dist-test-slave-2w3w krb5kdc[25204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:23:56 dist-test-slave-2w3w krb5kdc[25204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 11:23:56.852219 25650 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:23:56.839219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51094 (local address 127.16.197.62:42387)
0807 11:23:56.839495 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:56.839503 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:56.839899 (+ 396us) server_negotiation.cc:407] Connection header received
0807 11:23:56.840100 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:56.840105 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:56.840189 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:56.840278 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:56.842189 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.843021 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.844938 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.845435 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.845586 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:56.851930 (+ 6344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51094: BlockingRecv error: recv got EOF from 127.0.0.1:51094 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 07 11:23:56 dist-test-slave-2w3w krb5kdc[25204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM
I20260807 11:23:56.871295 25650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:56.855673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51104 (local address 127.16.197.62:42387)
0807 11:23:56.855870 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:56.855877 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:56.856093 (+ 216us) server_negotiation.cc:407] Connection header received
0807 11:23:56.856277 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:56.856280 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:56.856343 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:56.856422 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:56.857946 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.858717 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.860553 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:56.860978 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:56.861073 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:56.865772 (+ 4699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:56.865779 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:56.865783 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:56.865830 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:56.868523 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:56.869217 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:56.869223 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:56.869226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:56.869369 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:56.869782 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:56.869787 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:56.869792 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:56.870072 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:56.870333 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:56.870644 (+ 311us) server_negotiation.cc:299] Negotiation successful
0807 11:23:56.870877 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 11:23:56.875492 25248 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51104:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 11:23:56.876148 25248 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:23:56.882990 25395 tablet_service.cc:1467] Processing CreateTablet for tablet 75b0e0051c504206a479ce299fd4bbb1 (DEFAULT_TABLE table=test-table [id=a5691b0dd032408cbde3402f03ce702c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:56.883499 25395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75b0e0051c504206a479ce299fd4bbb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:56.883486 25697 tablet_service.cc:1467] Processing CreateTablet for tablet 75b0e0051c504206a479ce299fd4bbb1 (DEFAULT_TABLE table=test-table [id=a5691b0dd032408cbde3402f03ce702c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:56.883962 25697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75b0e0051c504206a479ce299fd4bbb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:56.887390 25547 tablet_service.cc:1467] Processing CreateTablet for tablet 75b0e0051c504206a479ce299fd4bbb1 (DEFAULT_TABLE table=test-table [id=a5691b0dd032408cbde3402f03ce702c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:56.887871 25547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75b0e0051c504206a479ce299fd4bbb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:56.888489 25800 tablet_bootstrap.cc:492] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7: Bootstrap starting.
I20260807 11:23:56.889811 25800 tablet_bootstrap.cc:654] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:56.892138 25800 tablet_bootstrap.cc:492] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7: No bootstrap required, opened a new log
I20260807 11:23:56.892293 25800 ts_tablet_manager.cc:1397] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260807 11:23:56.893337 25800 raft_consensus.cc:348] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:56.893608 25800 raft_consensus.cc:374] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:56.893712 25800 raft_consensus.cc:729] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fea1e3cfe29487f979dc0929a53cef7, State: Initialized, Role: FOLLOWER
I20260807 11:23:56.893997 25800 consensus_queue.cc:260] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:56.897126 25800 ts_tablet_manager.cc:1428] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260807 11:23:56.900962 25801 tablet_bootstrap.cc:492] T 75b0e0051c504206a479ce299fd4bbb1 P e5cb1ba24cf843f792962faf2870dbe7: Bootstrap starting.
I20260807 11:23:56.902817 25801 tablet_bootstrap.cc:654] T 75b0e0051c504206a479ce299fd4bbb1 P e5cb1ba24cf843f792962faf2870dbe7: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:56.902894 25877 tablet_bootstrap.cc:492] T 75b0e0051c504206a479ce299fd4bbb1 P b410615d35614d9db0b7ace7f1a19e8e: Bootstrap starting.
I20260807 11:23:56.904699 25877 tablet_bootstrap.cc:654] T 75b0e0051c504206a479ce299fd4bbb1 P b410615d35614d9db0b7ace7f1a19e8e: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:56.913947 25877 tablet_bootstrap.cc:492] T 75b0e0051c504206a479ce299fd4bbb1 P b410615d35614d9db0b7ace7f1a19e8e: No bootstrap required, opened a new log
I20260807 11:23:56.914217 25877 ts_tablet_manager.cc:1397] T 75b0e0051c504206a479ce299fd4bbb1 P b410615d35614d9db0b7ace7f1a19e8e: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.007s
I20260807 11:23:56.915266 25877 raft_consensus.cc:348] T 75b0e0051c504206a479ce299fd4bbb1 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:56.915632 25877 raft_consensus.cc:374] T 75b0e0051c504206a479ce299fd4bbb1 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:56.915789 25877 raft_consensus.cc:729] T 75b0e0051c504206a479ce299fd4bbb1 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b410615d35614d9db0b7ace7f1a19e8e, State: Initialized, Role: FOLLOWER
I20260807 11:23:56.916219 25877 consensus_queue.cc:260] T 75b0e0051c504206a479ce299fd4bbb1 P b410615d35614d9db0b7ace7f1a19e8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:56.916997 25877 ts_tablet_manager.cc:1428] T 75b0e0051c504206a479ce299fd4bbb1 P b410615d35614d9db0b7ace7f1a19e8e: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 11:23:56.918676 25801 tablet_bootstrap.cc:492] T 75b0e0051c504206a479ce299fd4bbb1 P e5cb1ba24cf843f792962faf2870dbe7: No bootstrap required, opened a new log
I20260807 11:23:56.919090 25801 ts_tablet_manager.cc:1397] T 75b0e0051c504206a479ce299fd4bbb1 P e5cb1ba24cf843f792962faf2870dbe7: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.004s
I20260807 11:23:56.920131 25801 raft_consensus.cc:348] T 75b0e0051c504206a479ce299fd4bbb1 P e5cb1ba24cf843f792962faf2870dbe7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:56.920400 25801 raft_consensus.cc:374] T 75b0e0051c504206a479ce299fd4bbb1 P e5cb1ba24cf843f792962faf2870dbe7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:56.920493 25801 raft_consensus.cc:729] T 75b0e0051c504206a479ce299fd4bbb1 P e5cb1ba24cf843f792962faf2870dbe7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5cb1ba24cf843f792962faf2870dbe7, State: Initialized, Role: FOLLOWER
I20260807 11:23:56.920773 25801 consensus_queue.cc:260] T 75b0e0051c504206a479ce299fd4bbb1 P e5cb1ba24cf843f792962faf2870dbe7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:56.922034 25801 ts_tablet_manager.cc:1428] T 75b0e0051c504206a479ce299fd4bbb1 P e5cb1ba24cf843f792962faf2870dbe7: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260807 11:23:57.034554 25807 raft_consensus.cc:482] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:23:57.034807 25807 raft_consensus.cc:504] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:57.036634 25807 leader_election.cc:288] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b410615d35614d9db0b7ace7f1a19e8e (127.16.197.2:36483), e5cb1ba24cf843f792962faf2870dbe7 (127.16.197.3:46859)
I20260807 11:23:57.044010 25795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:57.037154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:43727 (local address 127.16.197.3:46859)
0807 11:23:57.037392 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:57.037418 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:57.038178 (+ 760us) server_negotiation.cc:407] Connection header received
0807 11:23:57.038334 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:57.038339 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:57.038428 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:57.038535 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:57.039340 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:57.040023 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:57.041914 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:57.042944 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:57.043152 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:57.043207 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:57.043537 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 11:23:57.043674 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 11:23:57.044185 25881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:57.036858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:46859 (local address 127.16.197.1:43727)
0807 11:23:57.037955 (+ 1097us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:57.037985 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:57.038200 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:57.038555 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:57.038560 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:57.038629 (+ 69us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:57.039188 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:57.039201 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:57.040165 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:57.040171 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:57.041734 (+ 1563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:57.041747 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:57.043171 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:57.043177 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:57.043362 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:57.043402 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 11:23:57.043516 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 11:23:57.043669 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":960,"mutex_wait_us":4,"thread_start_us":887,"threads_started":1}
I20260807 11:23:57.044893 25793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:57.037216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:53317 (local address 127.16.197.2:36483)
0807 11:23:57.037479 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:57.037488 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:57.038123 (+ 635us) server_negotiation.cc:407] Connection header received
0807 11:23:57.038311 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:57.038316 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:57.038386 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:57.038495 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:23:57.039210 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:57.040021 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:57.042457 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:57.043578 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:57.043735 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:57.043813 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:57.044373 (+ 560us) server_negotiation.cc:299] Negotiation successful
0807 11:23:57.044537 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 11:23:57.045823 25880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:57.036858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:36483 (local address 127.16.197.1:53317)
0807 11:23:57.038001 (+ 1143us) negotiation.cc:107] Waiting for socket to connect
0807 11:23:57.038017 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 11:23:57.038143 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:23:57.038532 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:23:57.038537 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:23:57.038561 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:23:57.039001 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:57.039014 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:57.040178 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:57.040184 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:57.042243 (+ 2059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:23:57.042259 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:23:57.043837 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:23:57.043844 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:23:57.044046 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:57.044200 (+ 154us) client_negotiation.cc:769] Sending connection context
0807 11:23:57.045122 (+ 922us) client_negotiation.cc:240] Negotiation successful
0807 11:23:57.045330 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1049,"mutex_wait_us":62,"spinlock_wait_cycles":87424,"thread_start_us":709,"threads_started":1}
I20260807 11:23:57.046742 25568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75b0e0051c504206a479ce299fd4bbb1" candidate_uuid: "7fea1e3cfe29487f979dc0929a53cef7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b410615d35614d9db0b7ace7f1a19e8e" is_pre_election: true
I20260807 11:23:57.047173 25568 raft_consensus.cc:2454] T 75b0e0051c504206a479ce299fd4bbb1 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fea1e3cfe29487f979dc0929a53cef7 in term 0.
I20260807 11:23:57.047708 25342 leader_election.cc:302] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fea1e3cfe29487f979dc0929a53cef7, b410615d35614d9db0b7ace7f1a19e8e; no voters:
I20260807 11:23:57.048195 25807 raft_consensus.cc:2781] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:23:57.048331 25807 raft_consensus.cc:482] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:23:57.048465 25807 raft_consensus.cc:3037] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:57.049386 25718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75b0e0051c504206a479ce299fd4bbb1" candidate_uuid: "7fea1e3cfe29487f979dc0929a53cef7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5cb1ba24cf843f792962faf2870dbe7" is_pre_election: true
I20260807 11:23:57.049655 25718 raft_consensus.cc:2454] T 75b0e0051c504206a479ce299fd4bbb1 P e5cb1ba24cf843f792962faf2870dbe7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fea1e3cfe29487f979dc0929a53cef7 in term 0.
I20260807 11:23:57.051918 25807 raft_consensus.cc:504] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:57.052673 25807 leader_election.cc:288] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7 [CANDIDATE]: Term 1 election: Requested vote from peers b410615d35614d9db0b7ace7f1a19e8e (127.16.197.2:36483), e5cb1ba24cf843f792962faf2870dbe7 (127.16.197.3:46859)
I20260807 11:23:57.053224 25568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75b0e0051c504206a479ce299fd4bbb1" candidate_uuid: "7fea1e3cfe29487f979dc0929a53cef7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b410615d35614d9db0b7ace7f1a19e8e"
I20260807 11:23:57.053226 25718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75b0e0051c504206a479ce299fd4bbb1" candidate_uuid: "7fea1e3cfe29487f979dc0929a53cef7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5cb1ba24cf843f792962faf2870dbe7"
I20260807 11:23:57.053453 25718 raft_consensus.cc:3037] T 75b0e0051c504206a479ce299fd4bbb1 P e5cb1ba24cf843f792962faf2870dbe7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:57.053453 25568 raft_consensus.cc:3037] T 75b0e0051c504206a479ce299fd4bbb1 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:57.056701 25718 raft_consensus.cc:2454] T 75b0e0051c504206a479ce299fd4bbb1 P e5cb1ba24cf843f792962faf2870dbe7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fea1e3cfe29487f979dc0929a53cef7 in term 1.
I20260807 11:23:57.056701 25568 raft_consensus.cc:2454] T 75b0e0051c504206a479ce299fd4bbb1 P b410615d35614d9db0b7ace7f1a19e8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fea1e3cfe29487f979dc0929a53cef7 in term 1.
I20260807 11:23:57.057251 25344 leader_election.cc:302] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7fea1e3cfe29487f979dc0929a53cef7, b410615d35614d9db0b7ace7f1a19e8e, e5cb1ba24cf843f792962faf2870dbe7; no voters:
I20260807 11:23:57.057674 25807 raft_consensus.cc:2781] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:57.058121 25807 raft_consensus.cc:686] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7 [term 1 LEADER]: Becoming Leader. State: Replica: 7fea1e3cfe29487f979dc0929a53cef7, State: Running, Role: LEADER
I20260807 11:23:57.058756 25807 consensus_queue.cc:237] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:57.062414 25248 catalog_manager.cc:5495] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fea1e3cfe29487f979dc0929a53cef7 (127.16.197.1). New cstate: current_term: 1 leader_uuid: "7fea1e3cfe29487f979dc0929a53cef7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } health_report { overall_health: UNKNOWN } } }
I20260807 11:23:57.102837 25885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:57.097170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44062 (local address 127.16.197.1:35809)
0807 11:23:57.097744 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:57.097748 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:57.097769 (+ 21us) server_negotiation.cc:407] Connection header received
0807 11:23:57.097868 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:57.097872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:57.097968 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:57.098116 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:23:57.098466 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:57.099327 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:57.100611 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:57.101030 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:57.101123 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:57.101554 (+ 431us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:23:57.102181 (+ 627us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:23:57.102292 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:57.102366 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 11:23:57.102432 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":309,"threads_started":1}
I20260807 11:23:57.106503 25568 raft_consensus.cc:1261] T 75b0e0051c504206a479ce299fd4bbb1 P b410615d35614d9db0b7ace7f1a19e8e [term 1 FOLLOWER]: Refusing update from remote peer 7fea1e3cfe29487f979dc0929a53cef7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:57.106557 25718 raft_consensus.cc:1261] T 75b0e0051c504206a479ce299fd4bbb1 P e5cb1ba24cf843f792962faf2870dbe7 [term 1 FOLLOWER]: Refusing update from remote peer 7fea1e3cfe29487f979dc0929a53cef7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:57.107288 25882 consensus_queue.cc:1035] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:57.107640 25807 consensus_queue.cc:1035] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:57.129398 25250 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:23:57.129694 25250 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:23:57.133219 25250 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7065983468d844f2800fb30776891978: Sending DeleteTablet for 3 replicas of tablet 75b0e0051c504206a479ce299fd4bbb1
I20260807 11:23:57.134193 25697 tablet_service.cc:1514] Processing DeleteTablet for tablet 75b0e0051c504206a479ce299fd4bbb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:57 UTC) from {username='oryx', principal='oryx/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:53814
I20260807 11:23:57.134274 25547 tablet_service.cc:1514] Processing DeleteTablet for tablet 75b0e0051c504206a479ce299fd4bbb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:57 UTC) from {username='oryx', principal='oryx/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:41944
I20260807 11:23:57.134505 25863 tablet_replica.cc:330] T 75b0e0051c504206a479ce299fd4bbb1 P e5cb1ba24cf843f792962faf2870dbe7: stopping tablet replica
I20260807 11:23:57.134618 25862 tablet_replica.cc:330] T 75b0e0051c504206a479ce299fd4bbb1 P b410615d35614d9db0b7ace7f1a19e8e: stopping tablet replica
I20260807 11:23:57.134755 25863 raft_consensus.cc:2229] T 75b0e0051c504206a479ce299fd4bbb1 P e5cb1ba24cf843f792962faf2870dbe7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:57.134837 25862 raft_consensus.cc:2229] T 75b0e0051c504206a479ce299fd4bbb1 P b410615d35614d9db0b7ace7f1a19e8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:57.135038 25863 raft_consensus.cc:2258] T 75b0e0051c504206a479ce299fd4bbb1 P e5cb1ba24cf843f792962faf2870dbe7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:57.135103 25862 raft_consensus.cc:2258] T 75b0e0051c504206a479ce299fd4bbb1 P b410615d35614d9db0b7ace7f1a19e8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:57.136271 25863 ts_tablet_manager.cc:1905] T 75b0e0051c504206a479ce299fd4bbb1 P e5cb1ba24cf843f792962faf2870dbe7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:57.136750 25862 ts_tablet_manager.cc:1905] T 75b0e0051c504206a479ce299fd4bbb1 P b410615d35614d9db0b7ace7f1a19e8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:57.137087 25250 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51104:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 11:23:57.138069 25250 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:23:57.139182 25395 tablet_service.cc:1514] Processing DeleteTablet for tablet 75b0e0051c504206a479ce299fd4bbb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:57 UTC) from {username='oryx', principal='oryx/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:44048
I20260807 11:23:57.139556 25861 tablet_replica.cc:330] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7: stopping tablet replica
I20260807 11:23:57.139765 25861 raft_consensus.cc:2229] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7 [term 1 LEADER]: Raft consensus shutting down.
I20260807 11:23:57.140349 25861 raft_consensus.cc:2258] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:57.142463 25861 ts_tablet_manager.cc:1905] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:57.147090 25697 tablet_service.cc:1467] Processing CreateTablet for tablet 16d22243a1b24b6e8ef503b2f31ec742 (DEFAULT_TABLE table=test-table [id=08d38749906f4dbeaf4db5d2b0165d13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:57.147405 25547 tablet_service.cc:1467] Processing CreateTablet for tablet 16d22243a1b24b6e8ef503b2f31ec742 (DEFAULT_TABLE table=test-table [id=08d38749906f4dbeaf4db5d2b0165d13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:57.147608 25697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16d22243a1b24b6e8ef503b2f31ec742. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:57.147593 25862 ts_tablet_manager.cc:1918] T 75b0e0051c504206a479ce299fd4bbb1 P b410615d35614d9db0b7ace7f1a19e8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:23:57.147755 25862 log.cc:1196] T 75b0e0051c504206a479ce299fd4bbb1 P b410615d35614d9db0b7ace7f1a19e8e: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-1/wal/wals/75b0e0051c504206a479ce299fd4bbb1
I20260807 11:23:57.147854 25547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16d22243a1b24b6e8ef503b2f31ec742. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:57.148267 25862 ts_tablet_manager.cc:1939] T 75b0e0051c504206a479ce299fd4bbb1 P b410615d35614d9db0b7ace7f1a19e8e: Deleting consensus metadata
I20260807 11:23:57.151242 25233 catalog_manager.cc:4841] TS b410615d35614d9db0b7ace7f1a19e8e (127.16.197.2:36483): tablet 75b0e0051c504206a479ce299fd4bbb1 (table test-table [id=a5691b0dd032408cbde3402f03ce702c]) successfully deleted
I20260807 11:23:57.151455 25395 tablet_service.cc:1467] Processing CreateTablet for tablet 16d22243a1b24b6e8ef503b2f31ec742 (DEFAULT_TABLE table=test-table [id=08d38749906f4dbeaf4db5d2b0165d13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:23:57.151893 25395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16d22243a1b24b6e8ef503b2f31ec742. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:23:57.155759 25877 tablet_bootstrap.cc:492] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e: Bootstrap starting.
I20260807 11:23:57.157500 25877 tablet_bootstrap.cc:654] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:57.160023 25877 tablet_bootstrap.cc:492] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e: No bootstrap required, opened a new log
I20260807 11:23:57.160209 25877 ts_tablet_manager.cc:1397] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260807 11:23:57.161007 25877 raft_consensus.cc:348] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:57.161288 25877 raft_consensus.cc:374] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:57.161386 25877 raft_consensus.cc:729] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b410615d35614d9db0b7ace7f1a19e8e, State: Initialized, Role: FOLLOWER
I20260807 11:23:57.161644 25877 consensus_queue.cc:260] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:57.162232 25877 ts_tablet_manager.cc:1428] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260807 11:23:57.162308 25863 ts_tablet_manager.cc:1918] T 75b0e0051c504206a479ce299fd4bbb1 P e5cb1ba24cf843f792962faf2870dbe7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:23:57.162482 25863 log.cc:1196] T 75b0e0051c504206a479ce299fd4bbb1 P e5cb1ba24cf843f792962faf2870dbe7: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-2/wal/wals/75b0e0051c504206a479ce299fd4bbb1
I20260807 11:23:57.168179 25861 ts_tablet_manager.cc:1918] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:23:57.168390 25861 log.cc:1196] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-0/wal/wals/75b0e0051c504206a479ce299fd4bbb1
I20260807 11:23:57.168833 25861 ts_tablet_manager.cc:1939] T 75b0e0051c504206a479ce299fd4bbb1 P 7fea1e3cfe29487f979dc0929a53cef7: Deleting consensus metadata
I20260807 11:23:57.170029 25236 catalog_manager.cc:4841] TS 7fea1e3cfe29487f979dc0929a53cef7 (127.16.197.1:35809): tablet 75b0e0051c504206a479ce299fd4bbb1 (table test-table [id=a5691b0dd032408cbde3402f03ce702c]) successfully deleted
I20260807 11:23:57.170243 25800 tablet_bootstrap.cc:492] T 16d22243a1b24b6e8ef503b2f31ec742 P 7fea1e3cfe29487f979dc0929a53cef7: Bootstrap starting.
I20260807 11:23:57.162948 25863 ts_tablet_manager.cc:1939] T 75b0e0051c504206a479ce299fd4bbb1 P e5cb1ba24cf843f792962faf2870dbe7: Deleting consensus metadata
I20260807 11:23:57.164994 25801 tablet_bootstrap.cc:492] T 16d22243a1b24b6e8ef503b2f31ec742 P e5cb1ba24cf843f792962faf2870dbe7: Bootstrap starting.
I20260807 11:23:57.172107 25800 tablet_bootstrap.cc:654] T 16d22243a1b24b6e8ef503b2f31ec742 P 7fea1e3cfe29487f979dc0929a53cef7: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:57.173336 25801 tablet_bootstrap.cc:654] T 16d22243a1b24b6e8ef503b2f31ec742 P e5cb1ba24cf843f792962faf2870dbe7: Neither blocks nor log segments found. Creating new log.
I20260807 11:23:57.174692 25235 catalog_manager.cc:4841] TS e5cb1ba24cf843f792962faf2870dbe7 (127.16.197.3:46859): tablet 75b0e0051c504206a479ce299fd4bbb1 (table test-table [id=a5691b0dd032408cbde3402f03ce702c]) successfully deleted
I20260807 11:23:57.183094 25800 tablet_bootstrap.cc:492] T 16d22243a1b24b6e8ef503b2f31ec742 P 7fea1e3cfe29487f979dc0929a53cef7: No bootstrap required, opened a new log
I20260807 11:23:57.183321 25800 ts_tablet_manager.cc:1397] T 16d22243a1b24b6e8ef503b2f31ec742 P 7fea1e3cfe29487f979dc0929a53cef7: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.007s
I20260807 11:23:57.184413 25800 raft_consensus.cc:348] T 16d22243a1b24b6e8ef503b2f31ec742 P 7fea1e3cfe29487f979dc0929a53cef7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:57.184749 25800 raft_consensus.cc:374] T 16d22243a1b24b6e8ef503b2f31ec742 P 7fea1e3cfe29487f979dc0929a53cef7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:57.184851 25800 raft_consensus.cc:729] T 16d22243a1b24b6e8ef503b2f31ec742 P 7fea1e3cfe29487f979dc0929a53cef7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fea1e3cfe29487f979dc0929a53cef7, State: Initialized, Role: FOLLOWER
I20260807 11:23:57.185143 25800 consensus_queue.cc:260] T 16d22243a1b24b6e8ef503b2f31ec742 P 7fea1e3cfe29487f979dc0929a53cef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:57.185734 25800 ts_tablet_manager.cc:1428] T 16d22243a1b24b6e8ef503b2f31ec742 P 7fea1e3cfe29487f979dc0929a53cef7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260807 11:23:57.186128 25801 tablet_bootstrap.cc:492] T 16d22243a1b24b6e8ef503b2f31ec742 P e5cb1ba24cf843f792962faf2870dbe7: No bootstrap required, opened a new log
I20260807 11:23:57.186339 25801 ts_tablet_manager.cc:1397] T 16d22243a1b24b6e8ef503b2f31ec742 P e5cb1ba24cf843f792962faf2870dbe7: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.003s
I20260807 11:23:57.187274 25801 raft_consensus.cc:348] T 16d22243a1b24b6e8ef503b2f31ec742 P e5cb1ba24cf843f792962faf2870dbe7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:57.187543 25801 raft_consensus.cc:374] T 16d22243a1b24b6e8ef503b2f31ec742 P e5cb1ba24cf843f792962faf2870dbe7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:23:57.187635 25801 raft_consensus.cc:729] T 16d22243a1b24b6e8ef503b2f31ec742 P e5cb1ba24cf843f792962faf2870dbe7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5cb1ba24cf843f792962faf2870dbe7, State: Initialized, Role: FOLLOWER
I20260807 11:23:57.187942 25801 consensus_queue.cc:260] T 16d22243a1b24b6e8ef503b2f31ec742 P e5cb1ba24cf843f792962faf2870dbe7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:57.188599 25801 ts_tablet_manager.cc:1428] T 16d22243a1b24b6e8ef503b2f31ec742 P e5cb1ba24cf843f792962faf2870dbe7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 11:23:57.325412 25806 raft_consensus.cc:482] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:23:57.325644 25806 raft_consensus.cc:504] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:57.326336 25806 leader_election.cc:288] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7fea1e3cfe29487f979dc0929a53cef7 (127.16.197.1:35809), e5cb1ba24cf843f792962faf2870dbe7 (127.16.197.3:46859)
I20260807 11:23:57.327037 25718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16d22243a1b24b6e8ef503b2f31ec742" candidate_uuid: "b410615d35614d9db0b7ace7f1a19e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5cb1ba24cf843f792962faf2870dbe7" is_pre_election: true
I20260807 11:23:57.327049 25417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16d22243a1b24b6e8ef503b2f31ec742" candidate_uuid: "b410615d35614d9db0b7ace7f1a19e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fea1e3cfe29487f979dc0929a53cef7" is_pre_election: true
I20260807 11:23:57.327293 25718 raft_consensus.cc:2454] T 16d22243a1b24b6e8ef503b2f31ec742 P e5cb1ba24cf843f792962faf2870dbe7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b410615d35614d9db0b7ace7f1a19e8e in term 0.
I20260807 11:23:57.327351 25417 raft_consensus.cc:2454] T 16d22243a1b24b6e8ef503b2f31ec742 P 7fea1e3cfe29487f979dc0929a53cef7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b410615d35614d9db0b7ace7f1a19e8e in term 0.
I20260807 11:23:57.327875 25494 leader_election.cc:302] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b410615d35614d9db0b7ace7f1a19e8e, e5cb1ba24cf843f792962faf2870dbe7; no voters:
I20260807 11:23:57.328203 25806 raft_consensus.cc:2781] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:23:57.328311 25806 raft_consensus.cc:482] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:23:57.328433 25806 raft_consensus.cc:3037] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:57.330426 25806 raft_consensus.cc:504] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:57.331151 25806 leader_election.cc:288] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e [CANDIDATE]: Term 1 election: Requested vote from peers 7fea1e3cfe29487f979dc0929a53cef7 (127.16.197.1:35809), e5cb1ba24cf843f792962faf2870dbe7 (127.16.197.3:46859)
I20260807 11:23:57.331697 25417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16d22243a1b24b6e8ef503b2f31ec742" candidate_uuid: "b410615d35614d9db0b7ace7f1a19e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fea1e3cfe29487f979dc0929a53cef7"
I20260807 11:23:57.331701 25718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16d22243a1b24b6e8ef503b2f31ec742" candidate_uuid: "b410615d35614d9db0b7ace7f1a19e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5cb1ba24cf843f792962faf2870dbe7"
I20260807 11:23:57.331913 25718 raft_consensus.cc:3037] T 16d22243a1b24b6e8ef503b2f31ec742 P e5cb1ba24cf843f792962faf2870dbe7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:57.331925 25417 raft_consensus.cc:3037] T 16d22243a1b24b6e8ef503b2f31ec742 P 7fea1e3cfe29487f979dc0929a53cef7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:23:57.334142 25417 raft_consensus.cc:2454] T 16d22243a1b24b6e8ef503b2f31ec742 P 7fea1e3cfe29487f979dc0929a53cef7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b410615d35614d9db0b7ace7f1a19e8e in term 1.
I20260807 11:23:57.334662 25495 leader_election.cc:302] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fea1e3cfe29487f979dc0929a53cef7, b410615d35614d9db0b7ace7f1a19e8e; no voters:
I20260807 11:23:57.334777 25718 raft_consensus.cc:2454] T 16d22243a1b24b6e8ef503b2f31ec742 P e5cb1ba24cf843f792962faf2870dbe7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b410615d35614d9db0b7ace7f1a19e8e in term 1.
I20260807 11:23:57.335021 25806 raft_consensus.cc:2781] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:23:57.335614 25806 raft_consensus.cc:686] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e [term 1 LEADER]: Becoming Leader. State: Replica: b410615d35614d9db0b7ace7f1a19e8e, State: Running, Role: LEADER
I20260807 11:23:57.335917 25806 consensus_queue.cc:237] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } }
I20260807 11:23:57.338960 25250 catalog_manager.cc:5495] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e reported cstate change: term changed from 0 to 1, leader changed from <none> to b410615d35614d9db0b7ace7f1a19e8e (127.16.197.2). New cstate: current_term: 1 leader_uuid: "b410615d35614d9db0b7ace7f1a19e8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b410615d35614d9db0b7ace7f1a19e8e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 36483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 } health_report { overall_health: UNKNOWN } } }
I20260807 11:23:57.381242 25793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:57.367778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41958 (local address 127.16.197.2:36483)
0807 11:23:57.368451 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:57.368459 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:57.368479 (+ 20us) server_negotiation.cc:407] Connection header received
0807 11:23:57.368547 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:57.368553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:57.368643 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:57.368740 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:23:57.369275 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:57.370435 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:57.378529 (+ 8094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:57.379305 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:57.379470 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:57.380254 (+ 784us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:23:57.380541 (+ 287us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:23:57.380634 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:57.380786 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 11:23:57.380929 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532}
I20260807 11:23:57.394752 25417 raft_consensus.cc:1261] T 16d22243a1b24b6e8ef503b2f31ec742 P 7fea1e3cfe29487f979dc0929a53cef7 [term 1 FOLLOWER]: Refusing update from remote peer b410615d35614d9db0b7ace7f1a19e8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:57.394796 25718 raft_consensus.cc:1261] T 16d22243a1b24b6e8ef503b2f31ec742 P e5cb1ba24cf843f792962faf2870dbe7 [term 1 FOLLOWER]: Refusing update from remote peer b410615d35614d9db0b7ace7f1a19e8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:23:57.395443 25894 consensus_queue.cc:1035] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fea1e3cfe29487f979dc0929a53cef7" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 35809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:57.396485 25894 consensus_queue.cc:1035] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5cb1ba24cf843f792962faf2870dbe7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:23:57.507000 25248 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:23:57.507263 25248 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:23:57.511373 25248 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7065983468d844f2800fb30776891978: Sending DeleteTablet for 3 replicas of tablet 16d22243a1b24b6e8ef503b2f31ec742
I20260807 11:23:57.512385 25547 tablet_service.cc:1514] Processing DeleteTablet for tablet 16d22243a1b24b6e8ef503b2f31ec742 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:57 UTC) from {username='oryx', principal='oryx/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:41944
I20260807 11:23:57.512791 25395 tablet_service.cc:1514] Processing DeleteTablet for tablet 16d22243a1b24b6e8ef503b2f31ec742 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:57 UTC) from {username='oryx', principal='oryx/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:44048
I20260807 11:23:57.513142 25862 tablet_replica.cc:330] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e: stopping tablet replica
I20260807 11:23:57.513368 25862 raft_consensus.cc:2229] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e [term 1 LEADER]: Raft consensus shutting down.
I20260807 11:23:57.513775 25862 raft_consensus.cc:2258] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:57.514134 25861 tablet_replica.cc:330] T 16d22243a1b24b6e8ef503b2f31ec742 P 7fea1e3cfe29487f979dc0929a53cef7: stopping tablet replica
I20260807 11:23:57.514379 25861 raft_consensus.cc:2229] T 16d22243a1b24b6e8ef503b2f31ec742 P 7fea1e3cfe29487f979dc0929a53cef7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:57.514629 25861 raft_consensus.cc:2258] T 16d22243a1b24b6e8ef503b2f31ec742 P 7fea1e3cfe29487f979dc0929a53cef7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:57.516209 25861 ts_tablet_manager.cc:1905] T 16d22243a1b24b6e8ef503b2f31ec742 P 7fea1e3cfe29487f979dc0929a53cef7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:57.523731 25862 ts_tablet_manager.cc:1905] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:57.524442 25697 tablet_service.cc:1514] Processing DeleteTablet for tablet 16d22243a1b24b6e8ef503b2f31ec742 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:23:57 UTC) from {username='oryx', principal='oryx/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:53814
I20260807 11:23:57.526794 25861 ts_tablet_manager.cc:1918] T 16d22243a1b24b6e8ef503b2f31ec742 P 7fea1e3cfe29487f979dc0929a53cef7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 11:23:57.526989 25861 log.cc:1196] T 16d22243a1b24b6e8ef503b2f31ec742 P 7fea1e3cfe29487f979dc0929a53cef7: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-0/wal/wals/16d22243a1b24b6e8ef503b2f31ec742
I20260807 11:23:57.527187 25863 tablet_replica.cc:330] T 16d22243a1b24b6e8ef503b2f31ec742 P e5cb1ba24cf843f792962faf2870dbe7: stopping tablet replica
I20260807 11:23:57.527403 25863 raft_consensus.cc:2229] T 16d22243a1b24b6e8ef503b2f31ec742 P e5cb1ba24cf843f792962faf2870dbe7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:23:57.527536 25861 ts_tablet_manager.cc:1939] T 16d22243a1b24b6e8ef503b2f31ec742 P 7fea1e3cfe29487f979dc0929a53cef7: Deleting consensus metadata
I20260807 11:23:57.527642 25863 raft_consensus.cc:2258] T 16d22243a1b24b6e8ef503b2f31ec742 P e5cb1ba24cf843f792962faf2870dbe7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:23:57.528813 25236 catalog_manager.cc:4841] TS 7fea1e3cfe29487f979dc0929a53cef7 (127.16.197.1:35809): tablet 16d22243a1b24b6e8ef503b2f31ec742 (table test-table [id=08d38749906f4dbeaf4db5d2b0165d13]) successfully deleted
I20260807 11:23:57.529271 25863 ts_tablet_manager.cc:1905] T 16d22243a1b24b6e8ef503b2f31ec742 P e5cb1ba24cf843f792962faf2870dbe7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:23:57.529527 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 25326
I20260807 11:23:57.537400 25863 ts_tablet_manager.cc:1918] T 16d22243a1b24b6e8ef503b2f31ec742 P e5cb1ba24cf843f792962faf2870dbe7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 11:23:57.537607 25863 log.cc:1196] T 16d22243a1b24b6e8ef503b2f31ec742 P e5cb1ba24cf843f792962faf2870dbe7: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-2/wal/wals/16d22243a1b24b6e8ef503b2f31ec742
I20260807 11:23:57.538076 25863 ts_tablet_manager.cc:1939] T 16d22243a1b24b6e8ef503b2f31ec742 P e5cb1ba24cf843f792962faf2870dbe7: Deleting consensus metadata
I20260807 11:23:57.538926 25862 ts_tablet_manager.cc:1918] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 11:23:57.539258 25862 log.cc:1196] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786101765888975-17172-0/minicluster-data/ts-1/wal/wals/16d22243a1b24b6e8ef503b2f31ec742
I20260807 11:23:57.539415 25235 catalog_manager.cc:4841] TS e5cb1ba24cf843f792962faf2870dbe7 (127.16.197.3:46859): tablet 16d22243a1b24b6e8ef503b2f31ec742 (table test-table [id=08d38749906f4dbeaf4db5d2b0165d13]) successfully deleted
I20260807 11:23:57.539834 25862 ts_tablet_manager.cc:1939] T 16d22243a1b24b6e8ef503b2f31ec742 P b410615d35614d9db0b7ace7f1a19e8e: Deleting consensus metadata
I20260807 11:23:57.540975 25233 catalog_manager.cc:4841] TS b410615d35614d9db0b7ace7f1a19e8e (127.16.197.2:36483): tablet 16d22243a1b24b6e8ef503b2f31ec742 (table test-table [id=08d38749906f4dbeaf4db5d2b0165d13]) successfully deleted
Aug 07 11:23:57 dist-test-slave-2w3w krb5kdc[25204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.197.1@KRBTEST.COM
I20260807 11:23:57.562134 25885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:57.532391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44064 (local address 127.16.197.1:35809)
0807 11:23:57.532952 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:57.532959 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:57.541359 (+ 8400us) server_negotiation.cc:407] Connection header received
0807 11:23:57.541537 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:57.541545 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:57.541650 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:57.541770 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:57.544030 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:57.545390 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:57.547428 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:57.548333 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:57.548460 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:57.555041 (+ 6581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:57.555066 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:57.555093 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:57.555148 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:57.558932 (+ 3784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:57.559803 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:57.559835 (+ 32us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:57.559846 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:57.560016 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:57.560422 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:57.560427 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:57.560430 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:57.560826 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:57.561199 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:57.561462 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 11:23:57.561734 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419}
I20260807 11:23:57.562840 25465 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:57.700393 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 25326
I20260807 11:23:57.725406 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 25475
Aug 07 11:23:57 dist-test-slave-2w3w krb5kdc[25204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.197.2@KRBTEST.COM
I20260807 11:23:57.741297 25793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:57.726263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41962 (local address 127.16.197.2:36483)
0807 11:23:57.726470 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:57.726477 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:57.726498 (+ 21us) server_negotiation.cc:407] Connection header received
0807 11:23:57.726607 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:57.726611 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:57.726674 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:57.726785 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:57.728078 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:57.728838 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:57.730113 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:57.730560 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:57.730699 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:57.735274 (+ 4575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:57.735290 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:57.735307 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:57.735345 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:57.738176 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:57.739104 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:57.739115 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:57.739118 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:57.739248 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:57.739650 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:57.739657 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:57.739660 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:57.740040 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:57.740381 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:57.740666 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 11:23:57.740946 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 11:23:57.742046 25616 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:57.870162 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 25475
I20260807 11:23:57.896632 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 25626
Aug 07 11:23:57 dist-test-slave-2w3w krb5kdc[25204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.197.3@KRBTEST.COM
I20260807 11:23:57.913008 25907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:23:57.897378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53828 (local address 127.16.197.3:46859)
0807 11:23:57.897924 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0807 11:23:57.897929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:23:57.897947 (+ 18us) server_negotiation.cc:407] Connection header received
0807 11:23:57.898002 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:23:57.898005 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:23:57.898068 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:23:57.898171 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:23:57.899572 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:57.900289 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:57.901503 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:23:57.901912 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:23:57.902008 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:23:57.906333 (+ 4325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:23:57.906350 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:23:57.906368 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:23:57.906414 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:23:57.909820 (+ 3406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:57.910931 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:57.911059 (+ 128us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:57.911062 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:57.911214 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:23:57.911635 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:23:57.911640 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:23:57.911642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:23:57.912034 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:23:57.912293 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0807 11:23:57.912552 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 11:23:57.912730 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":339,"threads_started":1}
I20260807 11:23:57.913683 25767 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:58.040457 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 25626
I20260807 11:23:58.064524 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 25217
I20260807 11:23:58.065797 25294 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:23:58.186995 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 25217
2026-08-07T11:23:58Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7136 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:23:58 dist-test-slave-2w3w krb5kdc[25914](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:23:58 dist-test-slave-2w3w krb5kdc[25914](info): set up 2 sockets
Aug 07 11:23:58 dist-test-slave-2w3w krb5kdc[25914](info): commencing operation
krb5kdc: starting...
W20260807 11:23:59.330791 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:23:59 dist-test-slave-2w3w krb5kdc[25914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:23:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:23:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:23:59.595100 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:33975
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:37179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:33975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:23:59.935470 25928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:23:59.935806 25928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:23:59.935972 25928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:23:59.943379 25928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:23:59.943498 25928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:23:59.943600 25928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:23:59.943698 25928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:23:59.943789 25928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:23:59.954079 25928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:37179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:33975
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:33975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:23:59.955518 25928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:23:59.956907 25928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:23:59.969691 25935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:59.969740 25934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:23:59.969691 25937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:23:59.970389 25928 server_base.cc:1034] running on GCE node
I20260807 11:23:59.971360 25928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:23:59.973008 25928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:23:59.974257 25928 hybrid_clock.cc:648] HybridClock initialized: now 1786101839974202 us; error 45 us; skew 500 ppm
I20260807 11:23:59.974633 25928 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T11:24:00Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1857 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:24:00 dist-test-slave-2w3w krb5kdc[25943](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:24:00 dist-test-slave-2w3w krb5kdc[25943](info): set up 2 sockets
Aug 07 11:24:00 dist-test-slave-2w3w krb5kdc[25943](info): commencing operation
krb5kdc: starting...
W20260807 11:24:01.197508 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.007s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:24:01 dist-test-slave-2w3w krb5kdc[25943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:24:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:24:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:24:01.455298 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:45621
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:42867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:45621
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:24:01.794121 25957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:24:01.794512 25957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:24:01.794691 25957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:24:01.802199 25957 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:24:01.802407 25957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:24:01.802533 25957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:24:01.802645 25957 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:24:01.802736 25957 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:24:01.813031 25957 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:42867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:45621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:45621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:24:01.814582 25957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:24:01.816071 25957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:24:01.828819 25963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:01.828913 25962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:01.828946 25966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:24:01.830073 25957 server_base.cc:1034] running on GCE node
I20260807 11:24:01.831017 25957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:24:01.832664 25957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:24:01.833912 25957 hybrid_clock.cc:648] HybridClock initialized: now 1786101841833868 us; error 82 us; skew 500 ppm
I20260807 11:24:01.834285 25957 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:24:01 dist-test-slave-2w3w krb5kdc[25943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101841, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:24:01.842053 25957 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:24:01.843901 25957 webserver.cc:460] Webserver started at http://127.16.197.62:43415/ using document root <none> and password file <none>
I20260807 11:24:01.844683 25957 fs_manager.cc:359] Metadata directory not provided
I20260807 11:24:01.844839 25957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:24:01.845186 25957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:24:01.848441 25957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "44a697feeaf046ae86787dbba7a02b50"
format_stamp: "Formatted at 2026-08-07 11:24:01 on dist-test-slave-2w3w"
server_key: "6482a9c981862bfcd50c46a4e1386a01"
server_key_iv: "16c95c678d520e21f13dc58cdb2aba9e"
server_key_version: "encryptionkey@0"
I20260807 11:24:01.849249 25957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "44a697feeaf046ae86787dbba7a02b50"
format_stamp: "Formatted at 2026-08-07 11:24:01 on dist-test-slave-2w3w"
server_key: "6482a9c981862bfcd50c46a4e1386a01"
server_key_iv: "16c95c678d520e21f13dc58cdb2aba9e"
server_key_version: "encryptionkey@0"
I20260807 11:24:01.854804 25957 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 11:24:01.858551 25973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:01.860096 25957 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:24:01.860301 25957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "44a697feeaf046ae86787dbba7a02b50"
format_stamp: "Formatted at 2026-08-07 11:24:01 on dist-test-slave-2w3w"
server_key: "6482a9c981862bfcd50c46a4e1386a01"
server_key_iv: "16c95c678d520e21f13dc58cdb2aba9e"
server_key_version: "encryptionkey@0"
I20260807 11:24:01.860536 25957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:24:01.877449 25957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:24:01.882249 25957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:24:01.882575 25957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:24:01.903465 25957 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:45621
I20260807 11:24:01.903487 26031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:45621 every 8 connection(s)
I20260807 11:24:01.904997 25957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:24:01.909215 26032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:24:01.911121 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 25957
I20260807 11:24:01.911337 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:24:01.912077 17172 external_mini_cluster.cc:1428] Setting key 4ea883e3abac01d6ff266c8ecb12402b
I20260807 11:24:01.923202 26032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50: Bootstrap starting.
I20260807 11:24:01.927301 26032 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50: Neither blocks nor log segments found. Creating new log.
Aug 07 11:24:01 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:24:01.928747 26032 log.cc:824] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50: Log is configured to *not* fsync() on all Append() calls
I20260807 11:24:01.932632 26032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50: No bootstrap required, opened a new log
I20260807 11:24:01.938095 26035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:01.914195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50192 (local address 127.16.197.62:45621)
0807 11:24:01.915143 (+ 948us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:01.915160 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:01.915213 (+ 53us) server_negotiation.cc:407] Connection header received
0807 11:24:01.917692 (+ 2479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:01.917723 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:01.918250 (+ 527us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:01.918714 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:01.920156 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:01.922215 (+ 2059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:01.923651 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:01.924694 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:01.924913 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:01.929693 (+ 4780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:01.929707 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:01.929732 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:24:01.929788 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:01.933331 (+ 3543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:01.934336 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:01.934344 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:01.934351 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:01.934541 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:01.935064 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:01.935068 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:01.935070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:01.935683 (+ 613us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:01.935954 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:01.936449 (+ 495us) server_negotiation.cc:299] Negotiation successful
0807 11:24:01.936911 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":366,"threads_started":1}
I20260807 11:24:01.942639 26032 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44a697feeaf046ae86787dbba7a02b50" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45621 } }
I20260807 11:24:01.943166 26032 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:24:01.943310 26032 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44a697feeaf046ae86787dbba7a02b50, State: Initialized, Role: FOLLOWER
I20260807 11:24:01.943995 26032 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44a697feeaf046ae86787dbba7a02b50" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45621 } }
I20260807 11:24:01.944303 26032 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:24:01.944470 26032 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:24:01.944686 26032 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:24:01.947274 26032 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44a697feeaf046ae86787dbba7a02b50" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45621 } }
I20260807 11:24:01.947774 26032 leader_election.cc:302] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44a697feeaf046ae86787dbba7a02b50; no voters:
I20260807 11:24:01.948346 26032 leader_election.cc:288] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:24:01.948513 26037 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:24:01.949227 26037 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [term 1 LEADER]: Becoming Leader. State: Replica: 44a697feeaf046ae86787dbba7a02b50, State: Running, Role: LEADER
I20260807 11:24:01.949785 26037 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44a697feeaf046ae86787dbba7a02b50" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45621 } }
I20260807 11:24:01.950517 26032 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:24:01.953204 26039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44a697feeaf046ae86787dbba7a02b50. Latest consensus state: current_term: 1 leader_uuid: "44a697feeaf046ae86787dbba7a02b50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44a697feeaf046ae86787dbba7a02b50" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45621 } } }
I20260807 11:24:01.953680 26039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [sys.catalog]: This master's current role is: LEADER
I20260807 11:24:01.953231 26038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44a697feeaf046ae86787dbba7a02b50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44a697feeaf046ae86787dbba7a02b50" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45621 } } }
I20260807 11:24:01.955238 26038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [sys.catalog]: This master's current role is: LEADER
I20260807 11:24:01.962392 26043 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:24:01.967623 26043 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:24:01.977784 26043 catalog_manager.cc:1347] Generated new cluster ID: 828313b44d4143959ccf0c402482aa04
I20260807 11:24:01.977948 26043 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:24:01.985419 26043 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:24:01.986769 26043 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:24:01.995507 26043 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50: Generated new TSK 0
I20260807 11:24:01.996392 26043 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.197.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
I20260807 11:24:02.058952 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:0
--local_ip_for_outbound_sockets=127.16.197.1
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:45621
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:42867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:24:02.398937 26059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:24:02.399376 26059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:24:02.399549 26059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:24:02.407245 26059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:24:02.407441 26059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:24:02.407624 26059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:24:02.418151 26059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:42867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:45621
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:24:02.419724 26059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:24:02.421125 26059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:24:02.435559 26067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:02.435678 26065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:02.435637 26064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:24:02.436335 26059 server_base.cc:1034] running on GCE node
I20260807 11:24:02.436995 26059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:24:02.438071 26059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:24:02.439288 26059 hybrid_clock.cc:648] HybridClock initialized: now 1786101842439237 us; error 46 us; skew 500 ppm
I20260807 11:24:02.439596 26059 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:24:02 dist-test-slave-2w3w krb5kdc[25943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101842, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:24:02.442915 26059 init.cc:377] Logged in from keytab as kudu/127.16.197.1@KRBTEST.COM (short username kudu)
I20260807 11:24:02.444542 26059 webserver.cc:460] Webserver started at http://127.16.197.1:39421/ using document root <none> and password file <none>
I20260807 11:24:02.445341 26059 fs_manager.cc:359] Metadata directory not provided
I20260807 11:24:02.445477 26059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:24:02.445829 26059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:24:02.449050 26059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/data/instance:
uuid: "04dc076949bf49dc9b24bbe6532bfd78"
format_stamp: "Formatted at 2026-08-07 11:24:02 on dist-test-slave-2w3w"
server_key: "cfca1124d8ba1b82b66478c8e7c655b0"
server_key_iv: "fe1a486496d1909267ea3ce88bc276b7"
server_key_version: "encryptionkey@0"
I20260807 11:24:02.449901 26059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance:
uuid: "04dc076949bf49dc9b24bbe6532bfd78"
format_stamp: "Formatted at 2026-08-07 11:24:02 on dist-test-slave-2w3w"
server_key: "cfca1124d8ba1b82b66478c8e7c655b0"
server_key_iv: "fe1a486496d1909267ea3ce88bc276b7"
server_key_version: "encryptionkey@0"
I20260807 11:24:02.455641 26059 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 11:24:02.459389 26074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:02.460916 26059 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 11:24:02.461125 26059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "04dc076949bf49dc9b24bbe6532bfd78"
format_stamp: "Formatted at 2026-08-07 11:24:02 on dist-test-slave-2w3w"
server_key: "cfca1124d8ba1b82b66478c8e7c655b0"
server_key_iv: "fe1a486496d1909267ea3ce88bc276b7"
server_key_version: "encryptionkey@0"
I20260807 11:24:02.461352 26059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:24:02.487504 26059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:24:02.492125 26059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:24:02.492462 26059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:24:02.493539 26059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:24:02.495111 26059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:24:02.495213 26059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:02.495370 26059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:24:02.495445 26059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:02.530711 26059 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:39975
I20260807 11:24:02.530802 26195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:39975 every 8 connection(s)
I20260807 11:24:02.532092 26059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:24:02.537297 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 26059
I20260807 11:24:02.537515 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance
I20260807 11:24:02.538343 17172 external_mini_cluster.cc:1428] Setting key e5e03b0ef29031a89c4e52e2cdec7f9a
Aug 07 11:24:02 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101842, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:24:02.556546 26198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:02.534853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:34765 (local address 127.16.197.62:45621)
0807 11:24:02.535370 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:02.535374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:02.536247 (+ 873us) server_negotiation.cc:407] Connection header received
0807 11:24:02.538932 (+ 2685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:02.538937 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:02.539084 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:02.539222 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:02.541103 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:02.541815 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:02.544363 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:02.544773 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:02.544867 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:02.549525 (+ 4658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:02.549530 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:02.549533 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:24:02.549570 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:02.553047 (+ 3477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:02.553809 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:02.553818 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:02.553824 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:02.553982 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:02.554416 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:02.554423 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:02.554429 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:02.554832 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:02.555017 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:02.555770 (+ 753us) server_negotiation.cc:299] Negotiation successful
0807 11:24:02.556037 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":301,"threads_started":1}
I20260807 11:24:02.557165 26199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:02.535376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.1:34765)
0807 11:24:02.536083 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:02.536124 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:02.538610 (+ 2486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:02.539511 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:02.539523 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:02.540093 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:02.540915 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:02.540941 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:02.541987 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:02.541996 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:02.544197 (+ 2201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:02.544217 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:02.545006 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:02.545015 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:02.545370 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:02.546174 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:02.546207 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:02.549245 (+ 3038us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:24:02.553194 (+ 3949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:24:02.553201 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:24:02.553214 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:24:02.553654 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:24:02.554095 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:24:02.554098 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:24:02.554100 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:24:02.554293 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:24:02.555010 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:24:02.555025 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:24:02.555352 (+ 327us) client_negotiation.cc:769] Sending connection context
0807 11:24:02.555580 (+ 228us) client_negotiation.cc:240] Negotiation successful
0807 11:24:02.555985 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":277,"threads_started":1}
I20260807 11:24:02.558979 26196 heartbeater.cc:347] Connected to a master server at 127.16.197.62:45621
I20260807 11:24:02.559334 26196 heartbeater.cc:464] Registering TS with master...
I20260807 11:24:02.560207 26196 heartbeater.cc:511] Master 127.16.197.62:45621 requested a full tablet report, sending...
I20260807 11:24:02.562884 25993 ts_manager.cc:194] Registered new tserver with Master: 04dc076949bf49dc9b24bbe6532bfd78 (127.16.197.1:39975)
I20260807 11:24:02.565348 25993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.1@KRBTEST.COM'} at 127.16.197.1:34765
WARNING: no policy specified for kudu/127.16.197.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
I20260807 11:24:02.600205 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:0
--local_ip_for_outbound_sockets=127.16.197.2
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:45621
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:42867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:24:02.941525 26202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:24:02.941881 26202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:24:02.942070 26202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:24:02.949746 26202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:24:02.949941 26202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:24:02.950126 26202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:24:02.960723 26202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:42867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:45621
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:24:02.962179 26202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:24:02.963797 26202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:24:02.978521 26210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:02.978564 26208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:02.978606 26207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:24:02.979583 26202 server_base.cc:1034] running on GCE node
I20260807 11:24:02.980320 26202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:24:02.981396 26202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:24:02.982606 26202 hybrid_clock.cc:648] HybridClock initialized: now 1786101842982546 us; error 62 us; skew 500 ppm
I20260807 11:24:02.983000 26202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:24:02 dist-test-slave-2w3w krb5kdc[25943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101842, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:24:02.986503 26202 init.cc:377] Logged in from keytab as kudu/127.16.197.2@KRBTEST.COM (short username kudu)
I20260807 11:24:02.988236 26202 webserver.cc:460] Webserver started at http://127.16.197.2:43051/ using document root <none> and password file <none>
I20260807 11:24:02.989017 26202 fs_manager.cc:359] Metadata directory not provided
I20260807 11:24:02.989163 26202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:24:02.989528 26202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:24:02.992817 26202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/data/instance:
uuid: "4ed40b63af1249eea47a3c07239ff812"
format_stamp: "Formatted at 2026-08-07 11:24:02 on dist-test-slave-2w3w"
server_key: "6ffe3766fa01208e246ca09947ec590a"
server_key_iv: "453073934cf47fda9b944efae742fae9"
server_key_version: "encryptionkey@0"
I20260807 11:24:02.993685 26202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance:
uuid: "4ed40b63af1249eea47a3c07239ff812"
format_stamp: "Formatted at 2026-08-07 11:24:02 on dist-test-slave-2w3w"
server_key: "6ffe3766fa01208e246ca09947ec590a"
server_key_iv: "453073934cf47fda9b944efae742fae9"
server_key_version: "encryptionkey@0"
I20260807 11:24:02.999465 26202 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260807 11:24:03.003201 26217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:03.004611 26202 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 11:24:03.004817 26202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "4ed40b63af1249eea47a3c07239ff812"
format_stamp: "Formatted at 2026-08-07 11:24:02 on dist-test-slave-2w3w"
server_key: "6ffe3766fa01208e246ca09947ec590a"
server_key_iv: "453073934cf47fda9b944efae742fae9"
server_key_version: "encryptionkey@0"
I20260807 11:24:03.005053 26202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:24:03.021930 26202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:24:03.026648 26202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:24:03.026953 26202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:24:03.028036 26202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:24:03.029632 26202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:24:03.029757 26202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:03.029889 26202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:24:03.029961 26202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:03.074040 26202 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:36013
I20260807 11:24:03.074081 26336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:36013 every 8 connection(s)
I20260807 11:24:03.075495 26202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:24:03.077030 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 26202
I20260807 11:24:03.077226 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance
I20260807 11:24:03.077843 17172 external_mini_cluster.cc:1428] Setting key 45d41d4cd02b0aa40e468ab36dc67320
Aug 07 11:24:03 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101842, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:24:03.097806 26339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:03.078270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:49259 (local address 127.16.197.62:45621)
0807 11:24:03.079077 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:03.079084 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:03.080126 (+ 1042us) server_negotiation.cc:407] Connection header received
0807 11:24:03.082090 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:03.082095 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:03.082194 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:03.082332 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:03.083977 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:03.084718 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:03.086306 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:03.086764 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:03.086913 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:03.091336 (+ 4423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:03.091346 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:03.091352 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:24:03.091404 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:03.094185 (+ 2781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:03.095269 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:03.095273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:03.095275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:03.095406 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:03.095960 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:03.095963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:03.095965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:03.096244 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:03.096355 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:03.097341 (+ 986us) server_negotiation.cc:299] Negotiation successful
0807 11:24:03.097526 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":523,"threads_started":1}
I20260807 11:24:03.099555 26340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:03.079059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.2:49259)
0807 11:24:03.079965 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:03.080020 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:03.081803 (+ 1783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:03.082593 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:03.082602 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:03.083201 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:03.083796 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:03.083813 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:03.084894 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:03.084898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:03.086148 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:03.086161 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:03.087053 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:03.087061 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:03.087295 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:03.088036 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:03.088077 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:03.091042 (+ 2965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:24:03.094361 (+ 3319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:24:03.094374 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:24:03.094397 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:24:03.095139 (+ 742us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:24:03.095530 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:24:03.095537 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:24:03.095544 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:24:03.095822 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:24:03.096378 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:24:03.096387 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:24:03.096903 (+ 516us) client_negotiation.cc:769] Sending connection context
0807 11:24:03.097272 (+ 369us) client_negotiation.cc:240] Negotiation successful
0807 11:24:03.097915 (+ 643us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":389,"threads_started":1}
I20260807 11:24:03.101244 26337 heartbeater.cc:347] Connected to a master server at 127.16.197.62:45621
I20260807 11:24:03.101574 26337 heartbeater.cc:464] Registering TS with master...
I20260807 11:24:03.102326 26337 heartbeater.cc:511] Master 127.16.197.62:45621 requested a full tablet report, sending...
I20260807 11:24:03.103905 25993 ts_manager.cc:194] Registered new tserver with Master: 4ed40b63af1249eea47a3c07239ff812 (127.16.197.2:36013)
WARNING: no policy specified for kudu/127.16.197.3@KRBTEST.COM; defaulting to no policy
I20260807 11:24:03.104789 25993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.2@KRBTEST.COM'} at 127.16.197.2:49259
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
I20260807 11:24:03.139094 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:0
--local_ip_for_outbound_sockets=127.16.197.3
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:45621
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:42867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:24:03.481875 26344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:24:03.482225 26344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:24:03.482421 26344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:24:03.489764 26344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:24:03.489892 26344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:24:03.490008 26344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:24:03.500116 26344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:42867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:45621
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:24:03.501472 26344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:24:03.502893 26344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:24:03.516554 26351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:03.516781 26350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:03.516712 26353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:24:03.517863 26344 server_base.cc:1034] running on GCE node
I20260807 11:24:03.518464 26344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:24:03.519492 26344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:24:03.520730 26344 hybrid_clock.cc:648] HybridClock initialized: now 1786101843520684 us; error 27 us; skew 500 ppm
I20260807 11:24:03.521056 26344 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:24:03 dist-test-slave-2w3w krb5kdc[25943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101843, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:24:03.524469 26344 init.cc:377] Logged in from keytab as kudu/127.16.197.3@KRBTEST.COM (short username kudu)
I20260807 11:24:03.525924 26344 webserver.cc:460] Webserver started at http://127.16.197.3:36213/ using document root <none> and password file <none>
I20260807 11:24:03.526616 26344 fs_manager.cc:359] Metadata directory not provided
I20260807 11:24:03.526718 26344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:24:03.527055 26344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:24:03.530259 26344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/data/instance:
uuid: "ec1b7474d0194344b421308f6cec5cc8"
format_stamp: "Formatted at 2026-08-07 11:24:03 on dist-test-slave-2w3w"
server_key: "61f19aa2ad9003767956ae8128beb8bf"
server_key_iv: "849c53afa7e84dcc3eaf1280f55af8df"
server_key_version: "encryptionkey@0"
I20260807 11:24:03.531303 26344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance:
uuid: "ec1b7474d0194344b421308f6cec5cc8"
format_stamp: "Formatted at 2026-08-07 11:24:03 on dist-test-slave-2w3w"
server_key: "61f19aa2ad9003767956ae8128beb8bf"
server_key_iv: "849c53afa7e84dcc3eaf1280f55af8df"
server_key_version: "encryptionkey@0"
I20260807 11:24:03.536772 26344 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 11:24:03.541319 26360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:03.543416 26344 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 11:24:03.543792 26344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "ec1b7474d0194344b421308f6cec5cc8"
format_stamp: "Formatted at 2026-08-07 11:24:03 on dist-test-slave-2w3w"
server_key: "61f19aa2ad9003767956ae8128beb8bf"
server_key_iv: "849c53afa7e84dcc3eaf1280f55af8df"
server_key_version: "encryptionkey@0"
I20260807 11:24:03.544005 26344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:24:03.555913 26344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:24:03.560146 26344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:24:03.560520 26344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:24:03.561565 26344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:24:03.563061 26344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:24:03.563139 26344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:03.563313 26344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:24:03.563395 26344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:03.569192 26196 heartbeater.cc:503] Master 127.16.197.62:45621 was elected leader, sending a full tablet report...
I20260807 11:24:03.598217 26344 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:37175
I20260807 11:24:03.598244 26477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:37175 every 8 connection(s)
I20260807 11:24:03.599763 26344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:24:03.605146 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 26344
I20260807 11:24:03.605340 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance
I20260807 11:24:03.605811 17172 external_mini_cluster.cc:1428] Setting key 4bdbb08887ba295c537c84ab02949295
Aug 07 11:24:03 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101843, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:24:03.621614 26480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:03.602781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:42671 (local address 127.16.197.62:45621)
0807 11:24:03.603291 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:03.603296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:03.604363 (+ 1067us) server_negotiation.cc:407] Connection header received
0807 11:24:03.606396 (+ 2033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:03.606402 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:03.606515 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:03.606653 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:03.608530 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:03.609576 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:03.611296 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:03.611712 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:03.611815 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:03.615474 (+ 3659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:03.615480 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:03.615483 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:24:03.615520 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:03.618129 (+ 2609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:03.618900 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:03.618904 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:03.618907 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:03.619114 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:03.619553 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:03.619557 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:03.619560 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:03.619935 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:03.620084 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:03.620943 (+ 859us) server_negotiation.cc:299] Negotiation successful
0807 11:24:03.621228 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":269,"threads_started":1}
I20260807 11:24:03.622605 26481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:03.603445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.3:42671)
0807 11:24:03.604189 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:03.604227 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:03.606107 (+ 1880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:03.606937 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:03.606951 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:03.607568 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:03.608350 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:03.608377 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:03.609713 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:03.609717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:03.611158 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:03.611172 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:03.611911 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:03.611917 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:03.612132 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:03.612887 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:03.612912 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:03.615227 (+ 2315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:24:03.618294 (+ 3067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:24:03.618303 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:24:03.618318 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:24:03.618771 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:24:03.619213 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:24:03.619216 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:24:03.619218 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:24:03.619421 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:24:03.620056 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:24:03.620063 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:24:03.620528 (+ 465us) client_negotiation.cc:769] Sending connection context
0807 11:24:03.620771 (+ 243us) client_negotiation.cc:240] Negotiation successful
0807 11:24:03.621350 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":285,"threads_started":1}
I20260807 11:24:03.624045 26478 heartbeater.cc:347] Connected to a master server at 127.16.197.62:45621
I20260807 11:24:03.624447 26478 heartbeater.cc:464] Registering TS with master...
I20260807 11:24:03.625200 26478 heartbeater.cc:511] Master 127.16.197.62:45621 requested a full tablet report, sending...
I20260807 11:24:03.626737 25993 ts_manager.cc:194] Registered new tserver with Master: ec1b7474d0194344b421308f6cec5cc8 (127.16.197.3:37175)
I20260807 11:24:03.627641 25993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.3@KRBTEST.COM'} at 127.16.197.3:42671
I20260807 11:24:03.634260 17172 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.16.197.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.197.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.197.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.1.keytab.
Entry for principal oryx/127.16.197.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.1.keytab.
WARNING: no policy specified for oryx/127.16.197.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.197.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.197.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.2.keytab.
Entry for principal oryx/127.16.197.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.2.keytab.
WARNING: no policy specified for oryx/127.16.197.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.197.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.197.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.3.keytab.
Entry for principal oryx/127.16.197.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.3.keytab.
I20260807 11:24:03.793984 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 26059
Aug 07 11:24:03 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.1@KRBTEST.COM
I20260807 11:24:03.811746 26490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:03.795325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42876 (local address 127.16.197.1:39975)
0807 11:24:03.796046 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:03.796054 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:03.796084 (+ 30us) server_negotiation.cc:407] Connection header received
0807 11:24:03.796146 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:03.796153 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:03.796363 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:03.796520 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:03.798073 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:03.799263 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:03.800651 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:03.801261 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:03.801368 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:03.805540 (+ 4172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:03.805554 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:03.805569 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:24:03.805613 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:03.808903 (+ 3290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:03.809655 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:03.809663 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:03.809665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:03.809802 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:03.810207 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:03.810211 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:03.810213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:03.810572 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:03.810803 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:03.811127 (+ 324us) server_negotiation.cc:299] Negotiation successful
0807 11:24:03.811330 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":376,"threads_started":1}
I20260807 11:24:03.813117 26190 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:24:03.898305 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 26059
I20260807 11:24:03.921799 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 26202
Aug 07 11:24:03 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.2@KRBTEST.COM
I20260807 11:24:03.938410 26494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:03.922722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47244 (local address 127.16.197.2:36013)
0807 11:24:03.923523 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:03.923530 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:03.923558 (+ 28us) server_negotiation.cc:407] Connection header received
0807 11:24:03.923617 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:03.923623 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:03.923840 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:03.923967 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:03.925158 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:03.926162 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:03.927454 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:03.928007 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:03.928104 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:03.932487 (+ 4383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:03.932500 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:03.932516 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:24:03.932554 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:03.935808 (+ 3254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:03.936477 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:03.936484 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:03.936486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:03.936613 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:03.936931 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:03.936935 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:03.936937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:03.937278 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:03.937517 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:03.937779 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 11:24:03.937984 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":528,"threads_started":1}
I20260807 11:24:03.939867 26331 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:24:04.023978 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 26202
I20260807 11:24:04.042289 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 26344
Aug 07 11:24:04 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.3@KRBTEST.COM
I20260807 11:24:04.060875 26497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:04.043501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44890 (local address 127.16.197.3:37175)
0807 11:24:04.044088 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:04.044096 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:04.044126 (+ 30us) server_negotiation.cc:407] Connection header received
0807 11:24:04.044187 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:04.044193 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:04.044458 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:04.044613 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:04.046457 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:04.048120 (+ 1663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:04.049634 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:04.050203 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:04.050312 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:04.054875 (+ 4563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:04.054888 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:04.054903 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:24:04.054940 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:04.057853 (+ 2913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:04.058513 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:04.058520 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:04.058523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:04.058640 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:04.059016 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:04.059020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:04.059025 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:04.059385 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:04.059696 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:04.060277 (+ 581us) server_negotiation.cc:299] Negotiation successful
0807 11:24:04.060485 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":295,"threads_started":1}
I20260807 11:24:04.062155 26473 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:24:04.147156 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 26344
I20260807 11:24:04.167328 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 25957
I20260807 11:24:04.168737 26027 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:24:04.274631 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 25957
I20260807 11:24:04.298084 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:45621
--webserver_interface=127.16.197.62
--webserver_port=43415
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:42867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:45621
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:24:04.639809 26500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:24:04.640245 26500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:24:04.640455 26500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:24:04.648216 26500 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:24:04.648381 26500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:24:04.648504 26500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:24:04.648604 26500 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:24:04.648692 26500 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:24:04.659065 26500 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:42867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:45621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:45621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=43415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:24:04.660528 26500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:24:04.661929 26500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:24:04.674861 26505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:04.674821 26506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:04.675488 26509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:24:04.675906 26500 server_base.cc:1034] running on GCE node
I20260807 11:24:04.676708 26500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:24:04.678241 26500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:24:04.679538 26500 hybrid_clock.cc:648] HybridClock initialized: now 1786101844679469 us; error 56 us; skew 500 ppm
I20260807 11:24:04.679952 26500 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:24:04 dist-test-slave-2w3w krb5kdc[25943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101844, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:24:04.683359 26500 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:24:04.685135 26500 webserver.cc:460] Webserver started at http://127.16.197.62:43415/ using document root <none> and password file <none>
I20260807 11:24:04.685860 26500 fs_manager.cc:359] Metadata directory not provided
I20260807 11:24:04.685983 26500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:24:04.692111 26500 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 11:24:04.695222 26516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:04.696728 26500 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 11:24:04.696888 26500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "44a697feeaf046ae86787dbba7a02b50"
format_stamp: "Formatted at 2026-08-07 11:24:01 on dist-test-slave-2w3w"
server_key: "6482a9c981862bfcd50c46a4e1386a01"
server_key_iv: "16c95c678d520e21f13dc58cdb2aba9e"
server_key_version: "encryptionkey@0"
I20260807 11:24:04.697893 26500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:24:04.727461 26500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:24:04.731995 26500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:24:04.732252 26500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:24:04.752496 26500 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:45621
I20260807 11:24:04.752516 26570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:45621 every 8 connection(s)
I20260807 11:24:04.754115 26500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:24:04.761718 26571 sys_catalog.cc:263] Verifying existing consensus state
I20260807 11:24:04.764498 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 26500
I20260807 11:24:04.765219 26571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50: Bootstrap starting.
I20260807 11:24:04.765416 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:39975
--local_ip_for_outbound_sockets=127.16.197.1
--tserver_master_addrs=127.16.197.62:45621
--webserver_port=39421
--webserver_interface=127.16.197.1
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:42867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.16.197.1
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 11:24:04.789017 26571 log.cc:824] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50: Log is configured to *not* fsync() on all Append() calls
I20260807 11:24:04.802143 26571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 11:24:04.802842 26571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50: Bootstrap complete.
I20260807 11:24:04.816799 26571 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44a697feeaf046ae86787dbba7a02b50" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45621 } }
I20260807 11:24:04.818111 26571 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44a697feeaf046ae86787dbba7a02b50, State: Initialized, Role: FOLLOWER
I20260807 11:24:04.818867 26571 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44a697feeaf046ae86787dbba7a02b50" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45621 } }
I20260807 11:24:04.819186 26571 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:24:04.819312 26571 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:24:04.819499 26571 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [term 1 FOLLOWER]: Advancing to term 2
I20260807 11:24:04.825115 26571 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44a697feeaf046ae86787dbba7a02b50" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45621 } }
I20260807 11:24:04.825760 26571 leader_election.cc:302] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44a697feeaf046ae86787dbba7a02b50; no voters:
I20260807 11:24:04.826447 26571 leader_election.cc:288] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 11:24:04.826612 26575 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 11:24:04.827888 26575 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [term 2 LEADER]: Becoming Leader. State: Replica: 44a697feeaf046ae86787dbba7a02b50, State: Running, Role: LEADER
I20260807 11:24:04.828513 26575 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44a697feeaf046ae86787dbba7a02b50" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45621 } }
I20260807 11:24:04.828742 26571 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:24:04.831514 26576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "44a697feeaf046ae86787dbba7a02b50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44a697feeaf046ae86787dbba7a02b50" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45621 } } }
I20260807 11:24:04.831602 26577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44a697feeaf046ae86787dbba7a02b50. Latest consensus state: current_term: 2 leader_uuid: "44a697feeaf046ae86787dbba7a02b50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44a697feeaf046ae86787dbba7a02b50" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45621 } } }
I20260807 11:24:04.832005 26577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [sys.catalog]: This master's current role is: LEADER
I20260807 11:24:04.832006 26576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50 [sys.catalog]: This master's current role is: LEADER
I20260807 11:24:04.833280 26580 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:24:04.851833 26580 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:24:04.853433 26580 catalog_manager.cc:1259] Loaded cluster ID: 828313b44d4143959ccf0c402482aa04
I20260807 11:24:04.853618 26580 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:24:04.857884 26580 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:24:04.859609 26580 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 44a697feeaf046ae86787dbba7a02b50: Loaded TSK: 0
I20260807 11:24:04.861244 26580 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 11:24:05.145918 26573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:24:05.146209 26573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:24:05.146350 26573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:24:05.153998 26573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:24:05.154125 26573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:24:05.154337 26573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:24:05.164494 26573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:42867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.1.keytab
--principal=oryx/127.16.197.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:39975
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=39421
--tserver_master_addrs=127.16.197.62:45621
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:24:05.165952 26573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:24:05.167368 26573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:24:05.182642 26598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:05.182605 26599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:05.183118 26603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:24:05.184849 26573 server_base.cc:1034] running on GCE node
I20260807 11:24:05.185678 26573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:24:05.186995 26573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:24:05.188408 26573 hybrid_clock.cc:648] HybridClock initialized: now 1786101845188317 us; error 65 us; skew 500 ppm
I20260807 11:24:05.188885 26573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:24:05 dist-test-slave-2w3w krb5kdc[25943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101845, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:24:05.193326 26573 init.cc:377] Logged in from keytab as oryx/127.16.197.1@KRBTEST.COM (short username oryx)
I20260807 11:24:05.195416 26573 webserver.cc:460] Webserver started at http://127.16.197.1:39421/ using document root <none> and password file <none>
I20260807 11:24:05.196404 26573 fs_manager.cc:359] Metadata directory not provided
I20260807 11:24:05.196583 26573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:24:05.204197 26573 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260807 11:24:05.207773 26611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:05.209560 26573 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 11:24:05.209749 26573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "04dc076949bf49dc9b24bbe6532bfd78"
format_stamp: "Formatted at 2026-08-07 11:24:02 on dist-test-slave-2w3w"
server_key: "cfca1124d8ba1b82b66478c8e7c655b0"
server_key_iv: "fe1a486496d1909267ea3ce88bc276b7"
server_key_version: "encryptionkey@0"
I20260807 11:24:05.211233 26573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:24:05.232224 26573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:24:05.236584 26573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:24:05.236826 26573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:24:05.237812 26573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:24:05.239230 26573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:24:05.239328 26573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:05.239423 26573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:24:05.239478 26573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:05.273938 26573 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:39975
I20260807 11:24:05.273998 26733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:39975 every 8 connection(s)
I20260807 11:24:05.275982 26573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:24:05.277213 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 26573
I20260807 11:24:05.277876 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:36013
--local_ip_for_outbound_sockets=127.16.197.2
--tserver_master_addrs=127.16.197.62:45621
--webserver_port=43051
--webserver_interface=127.16.197.2
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:42867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.16.197.2
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 11:24:05 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:05 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 11:24:05.705196 26738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:24:05.705845 26738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:24:05.706091 26738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:24:05.725780 26738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:24:05.726032 26738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:24:05.726352 26738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:24:05.752389 26738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:42867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.2.keytab
--principal=oryx/127.16.197.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:36013
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=43051
--tserver_master_addrs=127.16.197.62:45621
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:24:05.754894 26738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:24:05.757417 26738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:24:05.777029 26748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:05.777100 26746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:05.777082 26745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:24:05.779217 26738 server_base.cc:1034] running on GCE node
I20260807 11:24:05.779902 26738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:24:05.781020 26738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:24:05.784363 26738 hybrid_clock.cc:648] HybridClock initialized: now 1786101845784312 us; error 148 us; skew 500 ppm
I20260807 11:24:05.784734 26738 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:24:05 dist-test-slave-2w3w krb5kdc[25943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101845, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:24:05.788340 26738 init.cc:377] Logged in from keytab as oryx/127.16.197.2@KRBTEST.COM (short username oryx)
I20260807 11:24:05.790124 26738 webserver.cc:460] Webserver started at http://127.16.197.2:43051/ using document root <none> and password file <none>
I20260807 11:24:05.791119 26738 fs_manager.cc:359] Metadata directory not provided
I20260807 11:24:05.791321 26738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:24:05.797961 26738 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 11:24:05.801594 26756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:05.803642 26738 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 11:24:05.803862 26738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "4ed40b63af1249eea47a3c07239ff812"
format_stamp: "Formatted at 2026-08-07 11:24:02 on dist-test-slave-2w3w"
server_key: "6ffe3766fa01208e246ca09947ec590a"
server_key_iv: "453073934cf47fda9b944efae742fae9"
server_key_version: "encryptionkey@0"
I20260807 11:24:05.804955 26738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:24:05.822192 26738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:24:05.826854 26738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:24:05.827183 26738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:24:05.828334 26738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:24:05.830103 26738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:24:05.830229 26738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:05.830377 26738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:24:05.830467 26738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:05.874413 26738 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:36013
I20260807 11:24:05.874471 26877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:36013 every 8 connection(s)
I20260807 11:24:05.876158 26738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:24:05.880573 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 26738
I20260807 11:24:05.881369 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:37175
--local_ip_for_outbound_sockets=127.16.197.3
--tserver_master_addrs=127.16.197.62:45621
--webserver_port=36213
--webserver_interface=127.16.197.3
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:42867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.16.197.3
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 11:24:05 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:05 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 11:24:06.342344 26883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:24:06.343134 26883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:24:06.343444 26883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:24:06.362882 26883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:24:06.363267 26883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:24:06.363642 26883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:24:06.389613 26883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:42867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/krb5kdc/oryx_127.16.197.3.keytab
--principal=oryx/127.16.197.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:37175
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=36213
--tserver_master_addrs=127.16.197.62:45621
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:24:06.392704 26883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:24:06.395460 26883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:24:06.423362 26889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:06.424314 26890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:06.429616 26892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:24:06.431360 26883 server_base.cc:1034] running on GCE node
I20260807 11:24:06.432641 26883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:24:06.434450 26883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:24:06.435861 26883 hybrid_clock.cc:648] HybridClock initialized: now 1786101846435767 us; error 75 us; skew 500 ppm
I20260807 11:24:06.436492 26883 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:24:06 dist-test-slave-2w3w krb5kdc[25943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101846, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.197.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:24:06.442759 26883 init.cc:377] Logged in from keytab as oryx/127.16.197.3@KRBTEST.COM (short username oryx)
I20260807 11:24:06.447293 26883 webserver.cc:460] Webserver started at http://127.16.197.3:36213/ using document root <none> and password file <none>
I20260807 11:24:06.448922 26883 fs_manager.cc:359] Metadata directory not provided
I20260807 11:24:06.449277 26883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:24:06.460481 26883 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.007s sys 0.000s
I20260807 11:24:06.465814 26899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:06.468391 26883 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260807 11:24:06.468700 26883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "ec1b7474d0194344b421308f6cec5cc8"
format_stamp: "Formatted at 2026-08-07 11:24:03 on dist-test-slave-2w3w"
server_key: "61f19aa2ad9003767956ae8128beb8bf"
server_key_iv: "849c53afa7e84dcc3eaf1280f55af8df"
server_key_version: "encryptionkey@0"
I20260807 11:24:06.470929 26883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:24:06.505439 26883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:24:06.513873 26883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:24:06.514344 26883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:24:06.516220 26883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:24:06.518877 26883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:24:06.519057 26883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260807 11:24:06.519219 26883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:24:06.519353 26883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:06.579731 26883 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:37175
I20260807 11:24:06.579857 27020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:37175 every 8 connection(s)
I20260807 11:24:06.582605 26883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:24:06.586264 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 26883
Aug 07 11:24:06 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:06 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:06.630316 27025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:06.587577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50200 (local address 127.16.197.62:45621)
0807 11:24:06.603446 (+ 15869us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:06.603452 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:06.603473 (+ 21us) server_negotiation.cc:407] Connection header received
0807 11:24:06.603530 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:06.603537 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:06.603615 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:06.603753 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:06.605920 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:06.607040 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:06.609045 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:06.610077 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:06.610272 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:06.614151 (+ 3879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:06.614162 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:06.614181 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:24:06.614230 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:06.617981 (+ 3751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:06.619467 (+ 1486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:06.619478 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:06.619488 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:06.619732 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:06.620394 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:06.620398 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:06.620400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:06.621160 (+ 760us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:06.621495 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:06.628375 (+ 6880us) server_negotiation.cc:299] Negotiation successful
0807 11:24:06.629123 (+ 748us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15715,"thread_start_us":15566,"threads_started":1}
W20260807 11:24:06.638526 26736 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.340s user 0.008s sys 0.012s
W20260807 11:24:06.640618 26737 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:05.284358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:60113 (local address 127.16.197.62:45621)
0807 11:24:05.286296 (+ 1938us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:05.286315 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:05.286370 (+ 55us) server_negotiation.cc:407] Connection header received
0807 11:24:05.289272 (+ 2902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:05.289286 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:05.289664 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:05.289980 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:05.292184 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:05.293440 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:05.296233 (+ 2793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:05.296949 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:05.297119 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:06.640110 (+1342991us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.1:60113: BlockingRecv error: recv got EOF from 127.16.197.1:60113 (error 108)
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":356,"threads_started":1}
I20260807 11:24:06.640976 26736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:05.284394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.1:60113)
0807 11:24:05.285690 (+ 1296us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:05.285757 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:05.288643 (+ 2886us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:05.290208 (+ 1565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:05.290221 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:05.291080 (+ 859us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:05.291959 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:05.291988 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:05.293641 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:05.293649 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:05.296039 (+ 2390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:05.296060 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:05.297229 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:05.297240 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:05.297618 (+ 378us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:05.298471 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:05.298511 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:06.639584 (+1341073us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":940,"thread_start_us":735,"threads_started":1}
W20260807 11:24:06.641894 26734 heartbeater.cc:644] Failed to heartbeat to 127.16.197.62:45621 (0 consecutive failures): Not authorized: Failed to ping master at 127.16.197.62:45621: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Aug 07 11:24:06 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:06 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:06.656930 26736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:06.642715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.1:38875)
0807 11:24:06.643043 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:06.643065 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:06.643199 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:06.643588 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:06.643593 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:06.643989 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:06.644162 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:06.644171 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:06.645263 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:06.645272 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:06.646616 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:06.646637 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:06.647992 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:06.648003 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:06.648262 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:06.649358 (+ 1096us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:06.649396 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:06.656414 (+ 7018us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172}
W20260807 11:24:06.657296 26737 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:06.642914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:38875 (local address 127.16.197.62:45621)
0807 11:24:06.643129 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:06.643136 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:06.643158 (+ 22us) server_negotiation.cc:407] Connection header received
0807 11:24:06.643370 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:06.643375 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:06.643441 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:06.643551 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:06.644335 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:06.645088 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:06.646790 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:06.647707 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:06.647871 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:06.656890 (+ 9019us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.1:38875: BlockingRecv error: recv got EOF from 127.16.197.1:38875 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 07 11:24:06 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:06 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 11:24:06.677172 26737 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:06.659048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:35725 (local address 127.16.197.62:45621)
0807 11:24:06.659538 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:06.659544 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:06.659560 (+ 16us) server_negotiation.cc:407] Connection header received
0807 11:24:06.659608 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:06.659612 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:06.659713 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:06.659879 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:06.662054 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:06.663781 (+ 1727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:06.666752 (+ 2971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:06.667879 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:06.668033 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:06.676707 (+ 8674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.1:35725: BlockingRecv error: recv got EOF from 127.16.197.1:35725 (error 108)
Metrics: {"server-negotiator.queue_time_us":353}
I20260807 11:24:06.678062 26736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:06.658479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.1:35725)
0807 11:24:06.659039 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:06.659070 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:06.659256 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:06.660344 (+ 1088us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:06.660353 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:06.661283 (+ 930us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:06.661793 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:06.661826 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:06.664112 (+ 2286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:06.664122 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:06.666478 (+ 2356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:06.666499 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:06.668479 (+ 1980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:06.668490 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:06.668826 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:06.669929 (+ 1103us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:06.669968 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:06.677621 (+ 7653us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":333}
W20260807 11:24:06.678572 26734 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 11:24:07.239445 26882 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.341s user 0.007s sys 0.014s
I20260807 11:24:07.241216 26882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:05.879729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.2:59089)
0807 11:24:05.881011 (+ 1282us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:05.881100 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:05.885173 (+ 4073us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:05.886756 (+ 1583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:05.886774 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:05.887821 (+ 1047us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:05.888806 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:05.888842 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:05.890870 (+ 2028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:05.890881 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:05.894461 (+ 3580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:05.894498 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:05.896203 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:05.896215 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:05.896771 (+ 556us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:05.898035 (+ 1264us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:05.898097 (+ 62us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:07.240278 (+1342181us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":774,"thread_start_us":462,"threads_started":1}
W20260807 11:24:07.241201 26881 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:05.879291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:59089 (local address 127.16.197.62:45621)
0807 11:24:05.879928 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:05.879935 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:05.885904 (+ 5969us) server_negotiation.cc:407] Connection header received
0807 11:24:05.886031 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:05.886040 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:05.886185 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:05.886357 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:05.889138 (+ 2781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:05.890506 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:05.894845 (+ 4339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:05.895750 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:05.895955 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:07.240739 (+1344784us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.2:59089: BlockingRecv error: recv got EOF from 127.16.197.2:59089 (error 108)
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":336,"threads_started":1}
W20260807 11:24:07.242339 26878 heartbeater.cc:644] Failed to heartbeat to 127.16.197.62:45621 (0 consecutive failures): Not authorized: Failed to ping master at 127.16.197.62:45621: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Aug 07 11:24:07 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:07 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:07.261559 26882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:07.243190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.2:51265)
0807 11:24:07.243518 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:07.243536 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:07.243643 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:07.244545 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:07.244554 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:07.245299 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:07.245678 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:07.245697 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:07.247029 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:07.247039 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:07.249447 (+ 2408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:07.249472 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:07.250454 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:07.250464 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:07.250766 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:07.256829 (+ 6063us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:07.256852 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:07.261230 (+ 4378us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202}
W20260807 11:24:07.261721 26881 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:07.243891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:51265 (local address 127.16.197.62:45621)
0807 11:24:07.244151 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:07.244156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:07.244172 (+ 16us) server_negotiation.cc:407] Connection header received
0807 11:24:07.244222 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:07.244226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:07.244288 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:07.244427 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:07.245918 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:07.246805 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:07.249695 (+ 2890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:07.250196 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:07.250332 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:07.261467 (+ 11135us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.2:51265: BlockingRecv error: recv got EOF from 127.16.197.2:51265 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 07 11:24:07 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:07 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:07.274031 26882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:07.262397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.2:59701)
0807 11:24:07.262563 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:07.262590 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:07.262717 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:07.263553 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:07.263557 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:07.263902 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:07.264081 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:07.264091 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:07.266153 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:07.266158 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:07.267349 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:07.267363 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:07.268716 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:07.268721 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:07.268877 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:07.269552 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:07.269571 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:07.273696 (+ 4125us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 11:24:07.274556 26878 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 11:24:07.275120 26881 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:07.262783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:59701 (local address 127.16.197.62:45621)
0807 11:24:07.263049 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:07.263059 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:07.263087 (+ 28us) server_negotiation.cc:407] Connection header received
0807 11:24:07.263169 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:07.263177 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:07.263310 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:07.263474 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:07.264249 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:07.265938 (+ 1689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:07.267514 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:07.268531 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:07.268717 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:07.274088 (+ 5371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.2:59701: BlockingRecv error: recv got EOF from 127.16.197.2:59701 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 11:24:07 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:07 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:07.689442 27028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:07.679290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.1:58295)
0807 11:24:07.679894 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:07.679910 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:07.680088 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:07.680485 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:07.680489 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:07.680987 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:07.681141 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:07.681151 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:07.682165 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:07.682170 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:07.683252 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:07.683265 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:07.684060 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:07.684066 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:07.684238 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:07.684929 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:07.684948 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:07.689045 (+ 4097us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":359,"threads_started":1}
W20260807 11:24:07.689530 26881 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:07.679387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:58295 (local address 127.16.197.62:45621)
0807 11:24:07.679662 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:07.679671 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:07.680012 (+ 341us) server_negotiation.cc:407] Connection header received
0807 11:24:07.680229 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:07.680234 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:07.680316 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:07.680456 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:07.681303 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:07.682008 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:07.683411 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:07.683844 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:07.684008 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:07.689261 (+ 5253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.1:58295: BlockingRecv error: recv got EOF from 127.16.197.1:58295 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260807 11:24:07.762305 27023 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.146s user 0.007s sys 0.019s
W20260807 11:24:07.763867 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:06.587160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:53297 (local address 127.16.197.62:45621)
0807 11:24:06.595651 (+ 8491us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:06.595662 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:06.597348 (+ 1686us) server_negotiation.cc:407] Connection header received
0807 11:24:06.602003 (+ 4655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:06.602014 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:06.602176 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:06.602316 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:06.605993 (+ 3677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:06.607179 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:06.610070 (+ 2891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:06.611768 (+ 1698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:06.611909 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:07.763626 (+1151717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.3:53297: BlockingRecv error: recv got EOF from 127.16.197.3:53297 (error 108)
Metrics: {"mutex_wait_us":15,"server-negotiator.queue_time_us":8242,"thread_start_us":8050,"threads_started":1}
I20260807 11:24:07.764448 27023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:06.586547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.3:53297)
0807 11:24:06.596969 (+ 10422us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:06.597078 (+ 109us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:06.601265 (+ 4187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:06.603309 (+ 2044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:06.603325 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:06.604417 (+ 1092us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:06.605729 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:06.605770 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:06.607404 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:06.607409 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:06.609864 (+ 2455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:06.609897 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:06.614239 (+ 4342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:06.614252 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:06.614788 (+ 536us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:06.615893 (+ 1105us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:06.615939 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:07.763347 (+1147408us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":9887,"thread_start_us":14872,"threads_started":1}
W20260807 11:24:07.765331 27021 heartbeater.cc:644] Failed to heartbeat to 127.16.197.62:45621 (0 consecutive failures): Not authorized: Failed to ping master at 127.16.197.62:45621: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Aug 07 11:24:07 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:07 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:07.774987 27023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:07.766013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.3:58435)
0807 11:24:07.766195 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:07.766212 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:07.766310 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:07.766620 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:07.766623 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:07.766990 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:07.767155 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:07.767163 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:07.768192 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:07.768195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:07.769149 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:07.769159 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:07.769808 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:07.769812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:07.769930 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:07.770534 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:07.770550 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:07.774619 (+ 4069us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":68}
W20260807 11:24:07.775264 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:07.766097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:58435 (local address 127.16.197.62:45621)
0807 11:24:07.766358 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:07.766362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:07.766391 (+ 29us) server_negotiation.cc:407] Connection header received
0807 11:24:07.766438 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:07.766441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:07.766500 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:07.766581 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:07.767285 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:07.768070 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:07.769267 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:07.769661 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:07.769751 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:07.774953 (+ 5202us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.3:58435: BlockingRecv error: recv got EOF from 127.16.197.3:58435 (error 108)
Metrics: {"server-negotiator.queue_time_us":157}
Aug 07 11:24:07 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:07 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:07.784835 27023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:07.775983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.3:51141)
0807 11:24:07.776155 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:07.776174 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:07.776275 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:07.776662 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:07.776668 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:07.777038 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:07.777198 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:07.777207 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:07.778232 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:07.778238 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:07.779391 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:07.779403 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:07.780231 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:07.780237 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:07.780452 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:07.781042 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:07.781060 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:07.784626 (+ 3566us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 11:24:07.785172 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:07.776077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:51141 (local address 127.16.197.62:45621)
0807 11:24:07.776237 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:07.776241 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:07.776257 (+ 16us) server_negotiation.cc:407] Connection header received
0807 11:24:07.776416 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:07.776422 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:07.776514 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:07.776620 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:07.777351 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:07.778084 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:07.779534 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:07.780072 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:07.780214 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:07.784848 (+ 4634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.3:51141: BlockingRecv error: recv got EOF from 127.16.197.3:51141 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260807 11:24:07.785285 27021 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 11:24:08 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:08 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:08.285775 27029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:08.275401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.2:48185)
0807 11:24:08.276196 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:08.276222 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:08.276365 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:08.276846 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:08.276882 (+ 36us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:08.277716 (+ 834us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:08.277885 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:08.277895 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:08.278936 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:08.278942 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:08.280006 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:08.280024 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:08.280782 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:08.280786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:08.280943 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:08.281645 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:08.281664 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:08.285406 (+ 3742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":679,"thread_start_us":539,"threads_started":1}
W20260807 11:24:08.285929 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:08.275519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:48185 (local address 127.16.197.62:45621)
0807 11:24:08.275745 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:08.275751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:08.276329 (+ 578us) server_negotiation.cc:407] Connection header received
0807 11:24:08.276565 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:08.276573 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:08.276686 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:08.276807 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:08.278054 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:08.278773 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:08.280164 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:08.280621 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:08.280724 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:08.285670 (+ 4946us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.2:48185: BlockingRecv error: recv got EOF from 127.16.197.2:48185 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 07 11:24:08 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:08 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:08.700014 27031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:08.690699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.1:33585)
0807 11:24:08.691465 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:08.691485 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:08.691645 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:08.692055 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:08.692058 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:08.692474 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:08.692663 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:08.692671 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:08.693696 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:08.693700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:08.694617 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:08.694628 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:08.695433 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:08.695437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:08.695564 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:08.696151 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:08.696168 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:08.699758 (+ 3590us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":516,"threads_started":1}
W20260807 11:24:08.700263 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:08.690833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:33585 (local address 127.16.197.62:45621)
0807 11:24:08.691069 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:08.691077 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:08.691605 (+ 528us) server_negotiation.cc:407] Connection header received
0807 11:24:08.691827 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:08.691832 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:08.691915 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:08.692007 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:08.692789 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:08.693559 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:08.694762 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:08.695271 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:08.695372 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:08.699981 (+ 4609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.1:33585: BlockingRecv error: recv got EOF from 127.16.197.1:33585 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 07 11:24:08 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:08 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:08.795694 27032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:08.786023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.3:47455)
0807 11:24:08.786704 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:08.786721 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:08.786845 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:08.787237 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:08.787241 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:08.787686 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:08.787839 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:08.787847 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:08.788852 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:08.788856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:08.789987 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:08.790000 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:08.790822 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:08.790828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:08.791050 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:08.791648 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:08.791665 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:08.795449 (+ 3784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":413,"threads_started":1}
W20260807 11:24:08.795934 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:08.786128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:47455 (local address 127.16.197.62:45621)
0807 11:24:08.786375 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:08.786382 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:08.786814 (+ 432us) server_negotiation.cc:407] Connection header received
0807 11:24:08.786996 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:08.787004 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:08.787096 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:08.787191 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:08.787974 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:08.788718 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:08.790137 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:08.790615 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:08.790769 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:08.795691 (+ 4922us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.3:47455: BlockingRecv error: recv got EOF from 127.16.197.3:47455 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 07 11:24:09 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:09 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:09.298627 27034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:09.286854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.2:34751)
0807 11:24:09.287479 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:09.287511 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:09.287629 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:09.288054 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:09.288058 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:09.288493 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:09.288701 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:09.288711 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:09.290475 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:09.290480 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:09.291544 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:09.291557 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:09.292776 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:09.292780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:09.292916 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:09.293625 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:09.293642 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:09.298244 (+ 4602us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":387,"threads_started":1}
W20260807 11:24:09.298753 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:09.287008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:34751 (local address 127.16.197.62:45621)
0807 11:24:09.287246 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:09.287256 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:09.287594 (+ 338us) server_negotiation.cc:407] Connection header received
0807 11:24:09.287752 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:09.287761 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:09.287864 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:09.287990 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:09.288903 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:09.290277 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:09.291716 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:09.292591 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:09.292759 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:09.298476 (+ 5717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.2:34751: BlockingRecv error: recv got EOF from 127.16.197.2:34751 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 07 11:24:09 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:09 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:09.711323 27035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:09.701085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.1:33099)
0807 11:24:09.701858 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:09.701884 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:09.702097 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:09.702542 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:09.702550 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:09.703163 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:09.703379 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:09.703396 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:09.704832 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:09.704840 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:09.705783 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:09.705795 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:09.706473 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:09.706478 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:09.706607 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:09.707268 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:09.707287 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:09.711053 (+ 3766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":446,"threads_started":1}
W20260807 11:24:09.711512 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:09.701449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:33099 (local address 127.16.197.62:45621)
0807 11:24:09.701707 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:09.701714 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:09.702055 (+ 341us) server_negotiation.cc:407] Connection header received
0807 11:24:09.702244 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:09.702250 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:09.702345 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:09.702494 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:09.703563 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:09.704693 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:09.705905 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:09.706344 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:09.706441 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:09.711277 (+ 4836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.1:33099: BlockingRecv error: recv got EOF from 127.16.197.1:33099 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 07 11:24:09 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:09 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:09.806618 27036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:09.796710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.3:42305)
0807 11:24:09.797210 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:09.797226 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:09.797349 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:09.797664 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:09.797667 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:09.798127 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:09.798324 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:09.798333 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:09.799590 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:09.799594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:09.800538 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:09.800550 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:09.801305 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:09.801310 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:09.801515 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:09.802199 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:09.802219 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:09.806356 (+ 4137us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":295,"threads_started":1}
W20260807 11:24:09.806789 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:09.796851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:42305 (local address 127.16.197.62:45621)
0807 11:24:09.797108 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:09.797114 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:09.797315 (+ 201us) server_negotiation.cc:407] Connection header received
0807 11:24:09.797454 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:09.797458 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:09.797537 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:09.797623 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:09.798488 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:09.799460 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:09.800745 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:09.801164 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:09.801265 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:09.806552 (+ 5287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.3:42305: BlockingRecv error: recv got EOF from 127.16.197.3:42305 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 07 11:24:10 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:10 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:10.309091 27037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:10.299570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.2:46463)
0807 11:24:10.300146 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:10.300168 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:10.300347 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:10.300713 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:10.300718 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:10.301114 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:10.301279 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:10.301288 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:10.302468 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:10.302478 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:10.303425 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:10.303437 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:10.304163 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:10.304169 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:10.304345 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:10.304947 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:10.304963 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:10.308741 (+ 3778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":321,"threads_started":1}
W20260807 11:24:10.309293 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:10.299666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:46463 (local address 127.16.197.62:45621)
0807 11:24:10.299972 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:10.299979 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:10.300261 (+ 282us) server_negotiation.cc:407] Connection header received
0807 11:24:10.300508 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:10.300512 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:10.300576 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:10.300679 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:10.301458 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:10.302309 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:10.303572 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:10.304013 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:10.304152 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:10.308975 (+ 4823us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.2:46463: BlockingRecv error: recv got EOF from 127.16.197.2:46463 (error 108)
Metrics: {"server-negotiator.queue_time_us":119}
Aug 07 11:24:10 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:10 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:10.722188 27038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:10.712523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.1:41575)
0807 11:24:10.713041 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:10.713059 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:10.713196 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:10.713516 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:10.713519 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:10.713917 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:10.714088 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:10.714096 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:10.715184 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:10.715191 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:10.716572 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:10.716585 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:10.717481 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:10.717486 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:10.717666 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:10.718252 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:10.718269 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:10.721815 (+ 3546us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":304,"threads_started":1}
W20260807 11:24:10.722388 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:10.712578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:41575 (local address 127.16.197.62:45621)
0807 11:24:10.712832 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:10.712842 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:10.713154 (+ 312us) server_negotiation.cc:407] Connection header received
0807 11:24:10.713309 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:10.713313 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:10.713378 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:10.713468 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:10.714227 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:10.715041 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:10.716733 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:10.717323 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:10.717458 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:10.722078 (+ 4620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.1:41575: BlockingRecv error: recv got EOF from 127.16.197.1:41575 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 07 11:24:10 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:10 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:10.817797 27039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:10.807713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.3:43457)
0807 11:24:10.808331 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:10.808354 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:10.808518 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:10.808977 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:10.808982 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:10.809386 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:10.809626 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:10.809636 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:10.810664 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:10.810672 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:10.811641 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:10.811652 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:10.812409 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:10.812415 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:10.812608 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:10.813241 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:10.813273 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:10.817557 (+ 4284us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":313,"threads_started":1}
W20260807 11:24:10.817992 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:10.807866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:43457 (local address 127.16.197.62:45621)
0807 11:24:10.808136 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:10.808144 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:10.808431 (+ 287us) server_negotiation.cc:407] Connection header received
0807 11:24:10.808667 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:10.808671 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:10.808822 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:10.808940 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:10.809804 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:10.810513 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:10.811818 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:10.812253 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:10.812391 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:10.817768 (+ 5377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.3:43457: BlockingRecv error: recv got EOF from 127.16.197.3:43457 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Aug 07 11:24:11 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:11 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:11.320056 27040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:11.310184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.2:42155)
0807 11:24:11.310748 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:11.310769 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:11.310892 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:11.311260 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:11.311264 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:11.311655 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:11.311852 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:11.311869 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:11.312953 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:11.312957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:11.313978 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:11.313990 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:11.314693 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:11.314697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:11.314858 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:11.315583 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:11.315601 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:11.319693 (+ 4092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":324,"threads_started":1}
W20260807 11:24:11.320223 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:11.310224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:42155 (local address 127.16.197.62:45621)
0807 11:24:11.310410 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:11.310415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:11.310861 (+ 446us) server_negotiation.cc:407] Connection header received
0807 11:24:11.311052 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:11.311057 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:11.311123 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:11.311214 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:11.312001 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:11.312815 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:11.314130 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:11.314558 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:11.314657 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:11.319929 (+ 5272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.2:42155: BlockingRecv error: recv got EOF from 127.16.197.2:42155 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 07 11:24:11 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:11 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:11.733654 27041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:11.723359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.1:48709)
0807 11:24:11.723935 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:11.723953 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:11.724072 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:11.724390 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:11.724393 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:11.724862 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:11.725021 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:11.725030 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:11.726027 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:11.726031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:11.727068 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:11.727081 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:11.727918 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:11.727926 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:11.728193 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:11.729107 (+ 914us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:11.729128 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:11.733404 (+ 4276us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":310,"threads_started":1}
W20260807 11:24:11.733837 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:11.723422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:48709 (local address 127.16.197.62:45621)
0807 11:24:11.723611 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:11.723615 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:11.724039 (+ 424us) server_negotiation.cc:407] Connection header received
0807 11:24:11.724176 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:11.724180 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:11.724244 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:11.724340 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:11.725163 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:11.725909 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:11.727195 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:11.727656 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:11.727818 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:11.733620 (+ 5802us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.1:48709: BlockingRecv error: recv got EOF from 127.16.197.1:48709 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 07 11:24:11 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:11 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:11.827862 27042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:11.818796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.3:55565)
0807 11:24:11.819404 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:11.819422 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:11.819534 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:11.819942 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:11.819946 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:11.820345 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:11.820522 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:11.820532 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:11.821533 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:11.821536 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:11.822535 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:11.822546 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:11.823276 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:11.823280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:11.823404 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:11.824055 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:11.824073 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:11.827599 (+ 3526us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":355,"threads_started":1}
W20260807 11:24:11.828136 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:11.818847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:55565 (local address 127.16.197.62:45621)
0807 11:24:11.819049 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:11.819053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:11.819527 (+ 474us) server_negotiation.cc:407] Connection header received
0807 11:24:11.819651 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:11.819655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:11.819748 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:11.819873 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:11.820642 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:11.821398 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:11.822656 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:11.823149 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:11.823246 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:11.827813 (+ 4567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.3:55565: BlockingRecv error: recv got EOF from 127.16.197.3:55565 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 07 11:24:12 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:12 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:12.334785 27043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:12.321001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.2:32905)
0807 11:24:12.321508 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:12.321528 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:12.321641 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:12.321948 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:12.321951 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:12.322375 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:12.322522 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:12.322530 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:12.323617 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:12.323622 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:12.325003 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:12.325016 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:12.325648 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:12.325652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:12.325769 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:12.331055 (+ 5286us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:12.331075 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:12.334541 (+ 3466us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":298,"threads_started":1}
W20260807 11:24:12.335047 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:12.321233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:32905 (local address 127.16.197.62:45621)
0807 11:24:12.321473 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:12.321480 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:12.321601 (+ 121us) server_negotiation.cc:407] Connection header received
0807 11:24:12.321739 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:12.321743 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:12.321806 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:12.321890 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:12.322640 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:12.323454 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:12.325124 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:12.325529 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:12.325622 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:12.334765 (+ 9143us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.2:32905: BlockingRecv error: recv got EOF from 127.16.197.2:32905 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 07 11:24:12 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:12 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:12.743542 27044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:12.734598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.1:50551)
0807 11:24:12.735212 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:12.735229 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:12.735351 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:12.735650 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:12.735653 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:12.736098 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:12.736274 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:12.736282 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:12.737269 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:12.737272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:12.738185 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:12.738195 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:12.738921 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:12.738925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:12.739139 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:12.739723 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:12.739740 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:12.743188 (+ 3448us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":343,"threads_started":1}
W20260807 11:24:12.743731 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:12.734707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:50551 (local address 127.16.197.62:45621)
0807 11:24:12.734912 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:12.734917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:12.735307 (+ 390us) server_negotiation.cc:407] Connection header received
0807 11:24:12.735456 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:12.735459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:12.735522 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:12.735606 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:12.736397 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:12.737145 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:12.738308 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:12.738768 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:12.738866 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:12.743430 (+ 4564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.1:50551: BlockingRecv error: recv got EOF from 127.16.197.1:50551 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 11:24:12 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:12 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:12.838836 27045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:12.828937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.3:36435)
0807 11:24:12.829516 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:12.829537 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:12.829693 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:12.830164 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:12.830168 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:12.830827 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:12.831060 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:12.831072 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:12.832447 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:12.832451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:12.833427 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:12.833440 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:12.834092 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:12.834096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:12.834220 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:12.834826 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:12.834860 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:12.838565 (+ 3705us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":282,"threads_started":1}
W20260807 11:24:12.839237 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:12.829076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:36435 (local address 127.16.197.62:45621)
0807 11:24:12.829316 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:12.829323 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:12.829637 (+ 314us) server_negotiation.cc:407] Connection header received
0807 11:24:12.829878 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:12.829884 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:12.830000 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:12.830107 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:12.831328 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:12.832291 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:12.833555 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:12.833963 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:12.834054 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:12.838807 (+ 4753us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.3:36435: BlockingRecv error: recv got EOF from 127.16.197.3:36435 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 07 11:24:13 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:13 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:13.345346 27046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:13.335903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.2:50489)
0807 11:24:13.336497 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:13.336526 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:13.336683 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:13.337103 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:13.337107 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:13.337665 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:13.337845 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:13.337855 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:13.338977 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:13.338981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:13.339904 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:13.339915 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:13.340607 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:13.340611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:13.340733 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:13.341410 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:13.341436 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:13.345094 (+ 3658us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":328,"threads_started":1}
W20260807 11:24:13.345678 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:13.335961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:50489 (local address 127.16.197.62:45621)
0807 11:24:13.336204 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:13.336211 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:13.336633 (+ 422us) server_negotiation.cc:407] Connection header received
0807 11:24:13.336829 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:13.336834 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:13.336931 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:13.337034 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:13.338024 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:13.338826 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:13.340044 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:13.340468 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:13.340566 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:13.345369 (+ 4803us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.2:50489: BlockingRecv error: recv got EOF from 127.16.197.2:50489 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 07 11:24:13 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:13 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:13.754922 27047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:13.744858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.1:46953)
0807 11:24:13.745477 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:13.745493 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:13.745671 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:13.746059 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:13.746065 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:13.746522 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:13.746697 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:13.746705 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:13.747786 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:13.747793 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:13.748823 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:13.748837 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:13.749830 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:13.749837 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:13.750011 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:13.750799 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:13.750817 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:13.754666 (+ 3849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":278,"threads_started":1}
W20260807 11:24:13.755220 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:13.744971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:46953 (local address 127.16.197.62:45621)
0807 11:24:13.745221 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:13.745229 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:13.745604 (+ 375us) server_negotiation.cc:407] Connection header received
0807 11:24:13.745813 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:13.745818 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:13.745903 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:13.746021 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:13.746856 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:13.747639 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:13.749008 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:13.749651 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:13.749790 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:13.754952 (+ 5162us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.1:46953: BlockingRecv error: recv got EOF from 127.16.197.1:46953 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 07 11:24:13 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:13 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:13.849530 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:13.839892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.3:50823)
0807 11:24:13.840436 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:13.840453 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:13.840615 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:13.841012 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:13.841019 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:13.841471 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:13.841633 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:13.841641 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:13.842713 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:13.842720 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:13.843696 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:13.843707 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:13.844508 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:13.844514 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:13.844703 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:13.845327 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:13.845345 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:13.849108 (+ 3763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":300,"threads_started":1}
W20260807 11:24:13.849715 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:13.839988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:50823 (local address 127.16.197.62:45621)
0807 11:24:13.840252 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:13.840261 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:13.840552 (+ 291us) server_negotiation.cc:407] Connection header received
0807 11:24:13.840745 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:13.840752 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:13.840834 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:13.840970 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:13.841789 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:13.842554 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:13.843877 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:13.844322 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:13.844464 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:13.849365 (+ 4901us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.3:50823: BlockingRecv error: recv got EOF from 127.16.197.3:50823 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Aug 07 11:24:14 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:14 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:14.356369 27049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:14.346342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.2:38343)
0807 11:24:14.347012 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:14.347035 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:14.347159 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:14.347486 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:14.347489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:14.347929 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:14.348106 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:14.348115 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:14.349154 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:14.349157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:14.350216 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:14.350229 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:14.351066 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:14.351070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:14.351189 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:14.351868 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:14.351888 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:14.356000 (+ 4112us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":307,"threads_started":1}
W20260807 11:24:14.356519 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:14.346454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:38343 (local address 127.16.197.62:45621)
0807 11:24:14.346643 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:14.346647 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:14.347115 (+ 468us) server_negotiation.cc:407] Connection header received
0807 11:24:14.347264 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:14.347267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:14.347333 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:14.347447 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:14.348263 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:14.349029 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:14.350357 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:14.350862 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:14.351011 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:14.356253 (+ 5242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.2:38343: BlockingRecv error: recv got EOF from 127.16.197.2:38343 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 11:24:14 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:14 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:14.766525 27050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:14.756245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.1:43167)
0807 11:24:14.757190 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:14.757207 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:14.757327 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:14.757672 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:14.757675 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:14.758093 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:14.758263 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:14.758272 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:14.759783 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:14.759788 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:14.760804 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:14.760817 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:14.761829 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:14.761833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:14.761958 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:14.762650 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:14.762669 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:14.766255 (+ 3586us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":793,"thread_start_us":664,"threads_started":1}
W20260807 11:24:14.766834 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:14.756327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:43167 (local address 127.16.197.62:45621)
0807 11:24:14.756507 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:14.756513 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:14.757306 (+ 793us) server_negotiation.cc:407] Connection header received
0807 11:24:14.757441 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:14.757446 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:14.757526 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:14.757620 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:14.758434 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:14.759637 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:14.760958 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:14.761643 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:14.761764 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:14.766510 (+ 4746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.1:43167: BlockingRecv error: recv got EOF from 127.16.197.1:43167 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 07 11:24:14 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:14 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:14.861052 27051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:14.850770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.3:59227)
0807 11:24:14.851560 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:14.851578 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:14.851721 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:14.852102 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:14.852105 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:14.852535 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:14.852713 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:14.852722 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:14.853887 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:14.853892 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:14.855308 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:14.855342 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:14.856107 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:14.856112 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:14.856237 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:14.856916 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:14.856945 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:14.860790 (+ 3845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":394,"threads_started":1}
W20260807 11:24:14.861275 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:14.850953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:59227 (local address 127.16.197.62:45621)
0807 11:24:14.851188 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:14.851192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:14.851660 (+ 468us) server_negotiation.cc:407] Connection header received
0807 11:24:14.851841 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:14.851845 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:14.851923 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:14.852017 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:14.852853 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:14.853684 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:14.855502 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:14.855957 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:14.856057 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:14.861034 (+ 4977us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.3:59227: BlockingRecv error: recv got EOF from 127.16.197.3:59227 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 07 11:24:15 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:15 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:15.367745 27052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:15.357659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.2:56859)
0807 11:24:15.358424 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:15.358452 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:15.358609 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:15.359018 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:15.359022 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:15.359547 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:15.359729 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:15.359739 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:15.361060 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:15.361066 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:15.362068 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:15.362081 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:15.362880 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:15.362886 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:15.363125 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:15.363826 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:15.363844 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:15.367479 (+ 3635us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":449,"threads_started":1}
W20260807 11:24:15.368109 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:15.357757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:56859 (local address 127.16.197.62:45621)
0807 11:24:15.357964 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:15.357973 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:15.358551 (+ 578us) server_negotiation.cc:407] Connection header received
0807 11:24:15.358749 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:15.358756 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:15.358836 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:15.358919 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:15.359916 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:15.360875 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:15.362228 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:15.362676 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:15.362823 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:15.367803 (+ 4980us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.2:56859: BlockingRecv error: recv got EOF from 127.16.197.2:56859 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 07 11:24:15 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:15 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:15.777647 27053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:15.767651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.1:52547)
0807 11:24:15.768244 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:15.768262 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:15.768387 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:15.768758 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:15.768761 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:15.769270 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:15.769436 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:15.769446 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:15.770514 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:15.770519 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:15.771623 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:15.771637 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:15.772344 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:15.772349 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:15.772487 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:15.773134 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:15.773154 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:15.777266 (+ 4112us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":319,"threads_started":1}
W20260807 11:24:15.777806 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:15.767729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:52547 (local address 127.16.197.62:45621)
0807 11:24:15.767917 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:15.767922 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:15.768347 (+ 425us) server_negotiation.cc:407] Connection header received
0807 11:24:15.768515 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:15.768518 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:15.768582 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:15.768687 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:15.769602 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:15.770375 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:15.771768 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:15.772192 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:15.772288 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:15.777526 (+ 5238us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.1:52547: BlockingRecv error: recv got EOF from 127.16.197.1:52547 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 11:24:15 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:15 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:15.872829 27054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:15.862244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.3:55529)
0807 11:24:15.863417 (+ 1173us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:15.863444 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:15.863611 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:15.864227 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:15.864233 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:15.864822 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:15.865053 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:15.865063 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:15.866069 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:15.866074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:15.867221 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:15.867235 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:15.867948 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:15.867952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:15.868086 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:15.868838 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:15.868871 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:15.872573 (+ 3702us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":946,"thread_start_us":804,"threads_started":1}
W20260807 11:24:15.873188 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:15.862415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:55529 (local address 127.16.197.62:45621)
0807 11:24:15.862670 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:15.862675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:15.863535 (+ 860us) server_negotiation.cc:407] Connection header received
0807 11:24:15.863891 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:15.863897 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:15.864002 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:15.864129 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:15.865189 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:15.865933 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:15.867368 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:15.867781 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:15.867885 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:15.872843 (+ 4958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.3:55529: BlockingRecv error: recv got EOF from 127.16.197.3:55529 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 07 11:24:16 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:16 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:16.377905 27055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:16.368817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.2:52687)
0807 11:24:16.369387 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:16.369410 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:16.369525 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:16.369843 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:16.369846 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:16.370257 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:16.370408 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:16.370417 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:16.371520 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:16.371525 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:16.372503 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:16.372516 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:16.373219 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:16.373223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:16.373350 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:16.374038 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:16.374056 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:16.377586 (+ 3530us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":299,"threads_started":1}
W20260807 11:24:16.378072 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:16.368895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:52687 (local address 127.16.197.62:45621)
0807 11:24:16.369076 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:16.369081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:16.369516 (+ 435us) server_negotiation.cc:407] Connection header received
0807 11:24:16.369630 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:16.369634 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:16.369698 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:16.369786 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:16.370535 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:16.371372 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:16.372639 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:16.373058 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:16.373201 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:16.377838 (+ 4637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.2:52687: BlockingRecv error: recv got EOF from 127.16.197.2:52687 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 11:24:16 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:16 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:16.789459 27056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:16.778982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.1:37891)
0807 11:24:16.779547 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:16.779564 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:16.779739 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:16.780143 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:16.780148 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:16.780679 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:16.780913 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:16.780927 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:16.782413 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:16.782423 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:16.783682 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:16.783697 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:16.784475 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:16.784480 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:16.784617 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:16.785299 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:16.785319 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:16.789151 (+ 3832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":324,"threads_started":1}
W20260807 11:24:16.789639 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:16.779065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:37891 (local address 127.16.197.62:45621)
0807 11:24:16.779321 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:16.779330 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:16.779667 (+ 337us) server_negotiation.cc:407] Connection header received
0807 11:24:16.779928 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:16.779932 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:16.779998 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:16.780107 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:16.781107 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:16.782192 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:16.783842 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:16.784283 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:16.784392 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:16.789405 (+ 5013us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.1:37891: BlockingRecv error: recv got EOF from 127.16.197.1:37891 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 07 11:24:16 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:16 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:16.883886 27058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:16.873914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.3:49015)
0807 11:24:16.874531 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:16.874548 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:16.874660 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:16.875052 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:16.875056 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:16.875526 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:16.875679 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:16.875687 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:16.877080 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:16.877084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:16.878035 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:16.878046 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:16.878789 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:16.878793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:16.878932 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:16.879719 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:16.879744 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:16.883642 (+ 3898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":376,"threads_started":1}
W20260807 11:24:16.884091 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:16.874012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:49015 (local address 127.16.197.62:45621)
0807 11:24:16.874374 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:16.874381 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:16.874630 (+ 249us) server_negotiation.cc:407] Connection header received
0807 11:24:16.874775 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:16.874778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:16.874842 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:16.874948 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:16.875830 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:16.876935 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:16.878191 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:16.878645 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:16.878746 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:16.883856 (+ 5110us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.3:49015: BlockingRecv error: recv got EOF from 127.16.197.3:49015 (error 108)
Metrics: {"server-negotiator.queue_time_us":200}
Aug 07 11:24:17 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:17 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:17.392970 27059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:17.378919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.2:53223)
0807 11:24:17.379553 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:17.379576 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:17.379756 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:17.380182 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:17.380188 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:17.380640 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:17.380799 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:17.380809 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:17.381946 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:17.381952 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:17.383134 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:17.383149 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:17.384035 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:17.384042 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:17.384251 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:17.388581 (+ 4330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:17.388603 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:17.392704 (+ 4101us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":312,"threads_started":1}
W20260807 11:24:17.393347 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:17.379128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:53223 (local address 127.16.197.62:45621)
0807 11:24:17.379381 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:17.379390 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:17.379673 (+ 283us) server_negotiation.cc:407] Connection header received
0807 11:24:17.379901 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:17.379908 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:17.379988 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:17.380152 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:17.380976 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:17.381771 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:17.383328 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:17.383846 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:17.384007 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:17.393010 (+ 9003us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.2:53223: BlockingRecv error: recv got EOF from 127.16.197.2:53223 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 07 11:24:17 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:17 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:17.800631 27060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:17.790570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.1:52355)
0807 11:24:17.791348 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:17.791366 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:17.791547 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:17.791961 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:17.791967 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:17.792574 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:17.792817 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:17.792833 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:17.794181 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:17.794188 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:17.795281 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:17.795293 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:17.796015 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:17.796020 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:17.796162 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:17.796821 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:17.796844 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:17.800390 (+ 3546us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":491,"threads_started":1}
W20260807 11:24:17.800871 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:17.790901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:52355 (local address 127.16.197.62:45621)
0807 11:24:17.791243 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:17.791250 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:17.791470 (+ 220us) server_negotiation.cc:407] Connection header received
0807 11:24:17.791695 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:17.791699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:17.791779 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:17.791897 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:17.793050 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:17.794022 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:17.795449 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:17.795860 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:17.795964 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:17.800622 (+ 4658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.1:52355: BlockingRecv error: recv got EOF from 127.16.197.1:52355 (error 108)
Metrics: {"server-negotiator.queue_time_us":141}
Aug 07 11:24:17 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:17 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:17.895072 27061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:17.884856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.3:57549)
0807 11:24:17.885460 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:17.885477 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:17.885645 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:17.886013 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:17.886017 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:17.886445 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:17.886615 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:17.886624 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:17.887892 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:17.887904 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:17.888904 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:17.888916 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:17.889684 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:17.889690 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:17.889879 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:17.890605 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:17.890626 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:17.894768 (+ 4142us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":334,"threads_started":1}
W20260807 11:24:17.895373 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:17.884949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:57549 (local address 127.16.197.62:45621)
0807 11:24:17.885180 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:17.885189 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:17.885581 (+ 392us) server_negotiation.cc:407] Connection header received
0807 11:24:17.885783 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:17.885793 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:17.885863 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:17.885977 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:17.886785 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:17.887741 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:17.889062 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:17.889516 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:17.889660 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:17.895054 (+ 5394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.3:57549: BlockingRecv error: recv got EOF from 127.16.197.3:57549 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 07 11:24:18 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:18 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:18.404238 27062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:18.394165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.2:58071)
0807 11:24:18.394669 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:18.394690 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:18.394835 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:18.395195 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:18.395199 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:18.395780 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:18.396014 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:18.396024 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:18.397074 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:18.397078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:18.398168 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:18.398181 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:18.399190 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:18.399197 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:18.399358 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:18.400107 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:18.400125 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:18.404004 (+ 3879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":270,"threads_started":1}
W20260807 11:24:18.404572 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:18.394286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:58071 (local address 127.16.197.62:45621)
0807 11:24:18.394563 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:18.394570 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:18.394803 (+ 233us) server_negotiation.cc:407] Connection header received
0807 11:24:18.394991 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:18.394995 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:18.395066 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:18.395148 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:18.396177 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:18.396950 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:18.398361 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:18.399017 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:18.399135 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:18.404248 (+ 5113us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.2:58071: BlockingRecv error: recv got EOF from 127.16.197.2:58071 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
Aug 07 11:24:18 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:18 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:18.812726 27063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:18.801936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.1:33599)
0807 11:24:18.802523 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:18.802541 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:18.802735 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:18.803206 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:18.803211 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:18.803701 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:18.803892 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:18.803902 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:18.805323 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:18.805327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:18.806620 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:18.806635 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:18.807382 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:18.807387 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:18.807538 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:18.808279 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:18.808299 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:18.812352 (+ 4053us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":335,"threads_started":1}
W20260807 11:24:18.812919 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:18.802037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:33599 (local address 127.16.197.62:45621)
0807 11:24:18.802282 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:18.802289 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:18.802651 (+ 362us) server_negotiation.cc:407] Connection header received
0807 11:24:18.802912 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:18.802918 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:18.803043 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:18.803165 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:18.804055 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:18.805125 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:18.806788 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:18.807235 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:18.807346 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:18.812613 (+ 5267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.1:33599: BlockingRecv error: recv got EOF from 127.16.197.1:33599 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 11:24:18 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:18 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:18.905782 27064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:18.896079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.3:59381)
0807 11:24:18.896646 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:18.896663 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:18.896780 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:18.897083 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:18.897086 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:18.897574 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:18.897767 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:18.897775 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:18.898774 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:18.898777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:18.899791 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:18.899804 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:18.900608 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:18.900614 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:18.900806 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:18.901488 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:18.901506 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:18.905457 (+ 3951us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":338,"threads_started":1}
W20260807 11:24:18.906104 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:18.896186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:59381 (local address 127.16.197.62:45621)
0807 11:24:18.896363 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:18.896368 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:18.896740 (+ 372us) server_negotiation.cc:407] Connection header received
0807 11:24:18.896882 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:18.896886 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:18.896950 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:18.897037 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:18.897910 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:18.898648 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:18.899923 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:18.900426 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:18.900575 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:18.905695 (+ 5120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.3:59381: BlockingRecv error: recv got EOF from 127.16.197.3:59381 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 07 11:24:19 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:19 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:19.415520 27065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:19.405529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.2:55807)
0807 11:24:19.406188 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:19.406210 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:19.406344 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:19.406744 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:19.406749 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:19.407269 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:19.407467 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:19.407477 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:19.408639 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:19.408646 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:19.409892 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:19.409910 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:19.410729 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:19.410733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:19.410887 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:19.411590 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:19.411608 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:19.415186 (+ 3578us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":391,"threads_started":1}
W20260807 11:24:19.415719 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:19.405639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:55807 (local address 127.16.197.62:45621)
0807 11:24:19.405821 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:19.405826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:19.406316 (+ 490us) server_negotiation.cc:407] Connection header received
0807 11:24:19.406473 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:19.406477 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:19.406548 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:19.406702 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:19.407642 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:19.408478 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:19.410041 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:19.410594 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:19.410695 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:19.415456 (+ 4761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.2:55807: BlockingRecv error: recv got EOF from 127.16.197.2:55807 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 07 11:24:19 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:19 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:19.823436 27066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:19.813900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.1:52319)
0807 11:24:19.814450 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:19.814467 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:19.814653 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:19.815077 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:19.815082 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:19.815536 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:19.815751 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:19.815760 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:19.816902 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:19.816905 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:19.817922 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:19.817934 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:19.818582 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:19.818586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:19.818708 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:19.819491 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:19.819509 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:19.823059 (+ 3550us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":320,"threads_started":1}
W20260807 11:24:19.823604 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:19.814046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:52319 (local address 127.16.197.62:45621)
0807 11:24:19.814287 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:19.814294 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:19.814555 (+ 261us) server_negotiation.cc:407] Connection header received
0807 11:24:19.814814 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:19.814818 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:19.814886 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:19.815026 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:19.815927 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:19.816750 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:19.818057 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:19.818453 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:19.818552 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:19.823299 (+ 4747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.1:52319: BlockingRecv error: recv got EOF from 127.16.197.1:52319 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 07 11:24:19 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:19 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:19.917567 27067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:19.906956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.3:46997)
0807 11:24:19.907621 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:19.907638 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:19.907818 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:19.908277 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:19.908281 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:19.908793 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:19.908976 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:19.908985 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:19.910463 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:19.910469 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:19.911725 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:19.911743 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:19.912597 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:19.912602 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:19.912735 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:19.913371 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:19.913399 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:19.917198 (+ 3799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":358,"threads_started":1}
W20260807 11:24:19.917809 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:19.907122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:46997 (local address 127.16.197.62:45621)
0807 11:24:19.907400 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:19.907411 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:19.907739 (+ 328us) server_negotiation.cc:407] Connection header received
0807 11:24:19.907956 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:19.907962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:19.908107 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:19.908230 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:19.909128 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:19.910240 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:19.911892 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:19.912416 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:19.912528 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:19.917522 (+ 4994us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.3:46997: BlockingRecv error: recv got EOF from 127.16.197.3:46997 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 07 11:24:20 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:20 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:20.426898 27068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:20.416703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.2:40801)
0807 11:24:20.417585 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:20.417615 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:20.417782 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:20.418279 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:20.418285 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:20.418864 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:20.419045 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:20.419056 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:20.420108 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:20.420112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:20.421060 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:20.421071 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:20.421836 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:20.421841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:20.421977 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:20.422625 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:20.422645 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:20.426638 (+ 3993us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":738,"thread_start_us":600,"threads_started":1}
W20260807 11:24:20.427268 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:20.416937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:40801 (local address 127.16.197.62:45621)
0807 11:24:20.417216 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:20.417225 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:20.417708 (+ 483us) server_negotiation.cc:407] Connection header received
0807 11:24:20.417959 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:20.417964 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:20.418058 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:20.418177 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:20.419193 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:20.419968 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:20.421202 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:20.421658 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:20.421754 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:20.427015 (+ 5261us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.2:40801: BlockingRecv error: recv got EOF from 127.16.197.2:40801 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 07 11:24:20 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:20 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.1@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:20.835755 27069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:20.825024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.1:44565)
0807 11:24:20.825574 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:20.825594 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:20.825728 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:20.826053 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:20.826056 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:20.826446 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:20.826640 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:20.826648 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:20.828079 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:20.828085 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:20.829507 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:20.829522 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:20.830651 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:20.830656 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:20.830824 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:20.831896 (+ 1072us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:20.831915 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:20.835502 (+ 3587us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":307,"threads_started":1}
W20260807 11:24:20.835987 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:20.825216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:44565 (local address 127.16.197.62:45621)
0807 11:24:20.825494 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:20.825501 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:20.825684 (+ 183us) server_negotiation.cc:407] Connection header received
0807 11:24:20.825845 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:20.825849 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:20.825915 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:20.826005 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:20.826791 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:20.827921 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:20.829725 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:20.830468 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:20.830583 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:20.835723 (+ 5140us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.1:44565: BlockingRecv error: recv got EOF from 127.16.197.1:44565 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 07 11:24:20 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:20 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.3@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:20.927702 27070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:20.918564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.3:60901)
0807 11:24:20.919120 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:20.919149 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:20.919316 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:20.919682 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:20.919687 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:20.920130 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:20.920275 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:20.920283 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:20.921389 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:20.921395 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:20.922318 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:20.922329 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:20.923050 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:20.923056 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:20.923231 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:20.923836 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:20.923854 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:20.927441 (+ 3587us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":264,"threads_started":1}
W20260807 11:24:20.928023 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:20.918671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:60901 (local address 127.16.197.62:45621)
0807 11:24:20.919091 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:20.919098 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:20.919236 (+ 138us) server_negotiation.cc:407] Connection header received
0807 11:24:20.919449 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:20.919459 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:20.919524 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:20.919630 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:20.920451 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:20.921240 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:20.922475 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:20.922877 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:20.923036 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:20.927707 (+ 4671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.3:60901: BlockingRecv error: recv got EOF from 127.16.197.3:60901 (error 108)
Metrics: {"server-negotiator.queue_time_us":249}
Aug 07 11:24:21 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 11:24:21 dist-test-slave-2w3w krb5kdc[25943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.197.2@KRBTEST.COM for oryx/127.16.197.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 11:24:21.438540 27071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:21.428286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45621 (local address 127.16.197.2:44921)
0807 11:24:21.429346 (+ 1060us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:21.429377 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:21.429522 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:21.430157 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:21.430161 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:21.430566 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:21.430750 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:21.430760 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:21.431867 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:21.431872 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:21.432835 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:21.432848 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:21.433593 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:21.433598 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:21.433746 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:21.434418 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:21.434437 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:21.438164 (+ 3727us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.197.62:45621: Server oryx/127.16.197.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":885,"thread_start_us":720,"threads_started":1}
W20260807 11:24:21.439070 27072 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:21.428445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:44921 (local address 127.16.197.62:45621)
0807 11:24:21.429844 (+ 1399us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:21.429850 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:21.429870 (+ 20us) server_negotiation.cc:407] Connection header received
0807 11:24:21.429927 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:21.429931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:21.429991 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:21.430081 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:21.430904 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:21.431697 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:21.432988 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:21.433423 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:21.433521 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:21.438429 (+ 4908us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.197.2:44921: BlockingRecv error: recv got EOF from 127.16.197.2:44921 (error 108)
Metrics: {"server-negotiator.queue_time_us":1254,"thread_start_us":1120,"threads_started":1}
I20260807 11:24:21.589738 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 26573
I20260807 11:24:21.603489 27074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:21.591072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41220 (local address 127.16.197.1:39975)
0807 11:24:21.591735 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:21.591742 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:21.591773 (+ 31us) server_negotiation.cc:407] Connection header received
0807 11:24:21.591838 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:21.591844 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:21.592102 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:21.592285 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:21.593861 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:21.595050 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:21.596276 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:21.596860 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:21.596979 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:21.599829 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:21.599838 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:21.599853 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:24:21.599915 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:21.602832 (+ 2917us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.16.197.1@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 11:24:21.603269 (+ 437us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41220: Request ticket server kudu/127.16.197.1@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":321,"threads_started":1}
W20260807 11:24:21.603621 27074 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41220: Request ticket server kudu/127.16.197.1@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 11:24:21.603693 17172 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 26573: Not authorized: Client connection negotiation failed: client connection to 127.16.197.1:39975: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.16.197.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 11:24:21.603981 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 26573
I20260807 11:24:21.624626 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 26738
I20260807 11:24:21.639448 27075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:21.625933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39934 (local address 127.16.197.2:36013)
0807 11:24:21.626658 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:21.626665 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:21.626696 (+ 31us) server_negotiation.cc:407] Connection header received
0807 11:24:21.626774 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:21.626782 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:21.627149 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:21.627329 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:21.629236 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:21.630509 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:21.631924 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:21.632520 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:21.632638 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:21.635616 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:21.635625 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:21.635639 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:24:21.635687 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:21.638555 (+ 2868us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.16.197.2@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 11:24:21.638896 (+ 341us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39934: Request ticket server kudu/127.16.197.2@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":383,"threads_started":1}
W20260807 11:24:21.639539 17172 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 26738: Not authorized: Client connection negotiation failed: client connection to 127.16.197.2:36013: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.16.197.2@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 11:24:21.639580 27075 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39934: Request ticket server kudu/127.16.197.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 11:24:21.639698 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 26738
I20260807 11:24:21.657410 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 26883
I20260807 11:24:21.672371 27076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:21.658715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55846 (local address 127.16.197.3:37175)
0807 11:24:21.659408 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:21.659416 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:21.659448 (+ 32us) server_negotiation.cc:407] Connection header received
0807 11:24:21.659518 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:21.659526 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:21.659807 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:21.660082 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:21.661826 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:21.663059 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:21.664549 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:21.665213 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:21.665400 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:21.668476 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:21.668486 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:21.668503 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:24:21.668571 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:21.671562 (+ 2991us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.16.197.3@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 11:24:21.671975 (+ 413us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55846: Request ticket server kudu/127.16.197.3@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":344,"threads_started":1}
W20260807 11:24:21.672400 17172 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 26883: Not authorized: Client connection negotiation failed: client connection to 127.16.197.3:37175: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.16.197.3@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 11:24:21.672497 27076 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55846: Request ticket server kudu/127.16.197.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 11:24:21.672591 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 26883
I20260807 11:24:21.690810 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 26500
I20260807 11:24:21.692386 26566 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:24:21.815392 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 26500
2026-08-07T11:24:21Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21774 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T11:24:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:24:21Z Disabled control of system clock
I20260807 11:24:21.906715 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:45363
--webserver_interface=127.16.197.62
--webserver_port=0
--builtin_ntp_servers=127.16.197.20:42831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:45363
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 11:24:22.270174 27081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:24:22.270516 27081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:24:22.270673 27081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:24:22.278537 27081 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:24:22.278676 27081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:24:22.278759 27081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:24:22.278849 27081 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:24:22.278996 27081 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:24:22.289018 27081 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:42831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:45363
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:45363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:24:22.290560 27081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:24:22.292631 27081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:24:22.306573 27088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:22.306663 27087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:22.306694 27090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:24:22.307389 27081 server_base.cc:1034] running on GCE node
I20260807 11:24:22.308300 27081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:24:22.309921 27081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:24:22.311167 27081 hybrid_clock.cc:648] HybridClock initialized: now 1786101862311132 us; error 62 us; skew 500 ppm
I20260807 11:24:22.311555 27081 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 11:24:22.314314 27081 webserver.cc:460] Webserver started at http://127.16.197.62:46055/ using document root <none> and password file <none>
I20260807 11:24:22.315199 27081 fs_manager.cc:359] Metadata directory not provided
I20260807 11:24:22.315368 27081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:24:22.315771 27081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:24:22.319115 27081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "39bdd2b49e694800b492cbe4bb3141fd"
format_stamp: "Formatted at 2026-08-07 11:24:22 on dist-test-slave-2w3w"
server_key: "f70d287f1e74abf40492687bac2759ce"
server_key_iv: "23bb1aa2c3b08f84350ad957b61d74fb"
server_key_version: "encryptionkey@0"
I20260807 11:24:22.320039 27081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "39bdd2b49e694800b492cbe4bb3141fd"
format_stamp: "Formatted at 2026-08-07 11:24:22 on dist-test-slave-2w3w"
server_key: "f70d287f1e74abf40492687bac2759ce"
server_key_iv: "23bb1aa2c3b08f84350ad957b61d74fb"
server_key_version: "encryptionkey@0"
I20260807 11:24:22.325913 27081 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 11:24:22.329986 27097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:22.331612 27081 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:24:22.331851 27081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "39bdd2b49e694800b492cbe4bb3141fd"
format_stamp: "Formatted at 2026-08-07 11:24:22 on dist-test-slave-2w3w"
server_key: "f70d287f1e74abf40492687bac2759ce"
server_key_iv: "23bb1aa2c3b08f84350ad957b61d74fb"
server_key_version: "encryptionkey@0"
I20260807 11:24:22.332129 27081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:24:22.374775 27081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:24:22.375804 27081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:24:22.376148 27081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:24:22.397289 27081 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:45363
I20260807 11:24:22.397329 27153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:45363 every 8 connection(s)
I20260807 11:24:22.398864 27081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:24:22.403281 27154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:24:22.405750 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 27081
I20260807 11:24:22.405969 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:24:22.406859 17172 external_mini_cluster.cc:1428] Setting key dd270255345e81de2eb84251860d73e4
I20260807 11:24:22.417414 27154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39bdd2b49e694800b492cbe4bb3141fd: Bootstrap starting.
I20260807 11:24:22.421182 27154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39bdd2b49e694800b492cbe4bb3141fd: Neither blocks nor log segments found. Creating new log.
I20260807 11:24:22.422320 27154 log.cc:824] T 00000000000000000000000000000000 P 39bdd2b49e694800b492cbe4bb3141fd: Log is configured to *not* fsync() on all Append() calls
I20260807 11:24:22.425662 27154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39bdd2b49e694800b492cbe4bb3141fd: No bootstrap required, opened a new log
I20260807 11:24:22.427676 27157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:22.409601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34354 (local address 127.16.197.62:45363)
0807 11:24:22.410856 (+ 1255us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:22.410870 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:22.410920 (+ 50us) server_negotiation.cc:407] Connection header received
0807 11:24:22.412836 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:22.412862 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:22.413407 (+ 545us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:22.413908 (+ 501us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:22.414620 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:22.416122 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:22.417584 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:22.418334 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:22.418578 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:22.424953 (+ 6375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:22.424962 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:22.424980 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:24:22.425588 (+ 608us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:22.425681 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:24:22.425706 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:22.425893 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:22.426165 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 11:24:22.426527 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":452,"threads_started":1}
I20260807 11:24:22.434526 27154 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39bdd2b49e694800b492cbe4bb3141fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bdd2b49e694800b492cbe4bb3141fd" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45363 } }
I20260807 11:24:22.435063 27154 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39bdd2b49e694800b492cbe4bb3141fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:24:22.435197 27154 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39bdd2b49e694800b492cbe4bb3141fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39bdd2b49e694800b492cbe4bb3141fd, State: Initialized, Role: FOLLOWER
I20260807 11:24:22.435925 27154 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39bdd2b49e694800b492cbe4bb3141fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bdd2b49e694800b492cbe4bb3141fd" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45363 } }
I20260807 11:24:22.436266 27154 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39bdd2b49e694800b492cbe4bb3141fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:24:22.436439 27154 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39bdd2b49e694800b492cbe4bb3141fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:24:22.436656 27154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39bdd2b49e694800b492cbe4bb3141fd [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:24:22.439208 27154 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39bdd2b49e694800b492cbe4bb3141fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bdd2b49e694800b492cbe4bb3141fd" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45363 } }
I20260807 11:24:22.439738 27154 leader_election.cc:302] T 00000000000000000000000000000000 P 39bdd2b49e694800b492cbe4bb3141fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39bdd2b49e694800b492cbe4bb3141fd; no voters:
I20260807 11:24:22.440394 27154 leader_election.cc:288] T 00000000000000000000000000000000 P 39bdd2b49e694800b492cbe4bb3141fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:24:22.440575 27159 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39bdd2b49e694800b492cbe4bb3141fd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:24:22.441548 27159 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39bdd2b49e694800b492cbe4bb3141fd [term 1 LEADER]: Becoming Leader. State: Replica: 39bdd2b49e694800b492cbe4bb3141fd, State: Running, Role: LEADER
I20260807 11:24:22.442222 27159 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39bdd2b49e694800b492cbe4bb3141fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bdd2b49e694800b492cbe4bb3141fd" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45363 } }
I20260807 11:24:22.442767 27154 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39bdd2b49e694800b492cbe4bb3141fd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:24:22.445744 27160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39bdd2b49e694800b492cbe4bb3141fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39bdd2b49e694800b492cbe4bb3141fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bdd2b49e694800b492cbe4bb3141fd" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45363 } } }
I20260807 11:24:22.445782 27161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39bdd2b49e694800b492cbe4bb3141fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39bdd2b49e694800b492cbe4bb3141fd. Latest consensus state: current_term: 1 leader_uuid: "39bdd2b49e694800b492cbe4bb3141fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39bdd2b49e694800b492cbe4bb3141fd" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45363 } } }
I20260807 11:24:22.446163 27160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39bdd2b49e694800b492cbe4bb3141fd [sys.catalog]: This master's current role is: LEADER
I20260807 11:24:22.446231 27161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39bdd2b49e694800b492cbe4bb3141fd [sys.catalog]: This master's current role is: LEADER
I20260807 11:24:22.448961 27168 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:24:22.454280 27168 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:24:22.463833 27168 catalog_manager.cc:1347] Generated new cluster ID: 5c5921608a86453ba93fb8aa09c0fece
I20260807 11:24:22.463972 27168 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:24:22.508071 27168 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:24:22.509608 27168 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:24:22.523125 27168 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39bdd2b49e694800b492cbe4bb3141fd: Generated new TSK 0
I20260807 11:24:22.524148 27168 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:24:22.533277 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:0
--local_ip_for_outbound_sockets=127.16.197.1
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:45363
--builtin_ntp_servers=127.16.197.20:42831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 11:24:22.887439 27180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:24:22.887761 27180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:24:22.887851 27180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:24:22.895752 27180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:24:22.895889 27180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:24:22.895999 27180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:24:22.906019 27180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:42831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:45363
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:24:22.907572 27180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:24:22.909598 27180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:24:22.924407 27186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:22.924212 27187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:22.924490 27189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:24:22.925570 27180 server_base.cc:1034] running on GCE node
I20260807 11:24:22.926373 27180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:24:22.927572 27180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:24:22.928833 27180 hybrid_clock.cc:648] HybridClock initialized: now 1786101862928767 us; error 62 us; skew 500 ppm
I20260807 11:24:22.929225 27180 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 11:24:22.931903 27180 webserver.cc:460] Webserver started at http://127.16.197.1:36649/ using document root <none> and password file <none>
I20260807 11:24:22.932624 27180 fs_manager.cc:359] Metadata directory not provided
I20260807 11:24:22.932718 27180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:24:22.933007 27180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:24:22.936205 27180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/data/instance:
uuid: "bba7808bf4ee49d2b505452b3efa977a"
format_stamp: "Formatted at 2026-08-07 11:24:22 on dist-test-slave-2w3w"
server_key: "5419b58375b9e28756b48dd6ed7d72a6"
server_key_iv: "7cc06a09c0aeafd593437f0e76b7536d"
server_key_version: "encryptionkey@0"
I20260807 11:24:22.937088 27180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance:
uuid: "bba7808bf4ee49d2b505452b3efa977a"
format_stamp: "Formatted at 2026-08-07 11:24:22 on dist-test-slave-2w3w"
server_key: "5419b58375b9e28756b48dd6ed7d72a6"
server_key_iv: "7cc06a09c0aeafd593437f0e76b7536d"
server_key_version: "encryptionkey@0"
I20260807 11:24:22.942795 27180 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 11:24:22.946681 27195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:22.948441 27180 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:24:22.948673 27180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "bba7808bf4ee49d2b505452b3efa977a"
format_stamp: "Formatted at 2026-08-07 11:24:22 on dist-test-slave-2w3w"
server_key: "5419b58375b9e28756b48dd6ed7d72a6"
server_key_iv: "7cc06a09c0aeafd593437f0e76b7536d"
server_key_version: "encryptionkey@0"
I20260807 11:24:22.948931 27180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:24:22.970868 27180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:24:22.971868 27180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:24:22.972213 27180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:24:22.973315 27180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:24:22.975190 27180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:24:22.975317 27180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:22.975463 27180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:24:22.975557 27180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:23.012199 27180 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:42773
I20260807 11:24:23.012261 27314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:42773 every 8 connection(s)
I20260807 11:24:23.013749 27180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:24:23.021540 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 27180
I20260807 11:24:23.021782 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance
I20260807 11:24:23.022501 17172 external_mini_cluster.cc:1428] Setting key 7e339fa95f93c8ad7c9ea7fcc757588c
I20260807 11:24:23.025588 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:0
--local_ip_for_outbound_sockets=127.16.197.2
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:45363
--builtin_ntp_servers=127.16.197.20:42831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 11:24:23.028563 27317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:23.016756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:47581 (local address 127.16.197.62:45363)
0807 11:24:23.017273 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:23.017278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:23.018473 (+ 1195us) server_negotiation.cc:407] Connection header received
0807 11:24:23.020521 (+ 2048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:23.020527 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:23.020637 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:23.020773 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:23.022237 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:23.023224 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:23.025175 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:23.025612 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:23.025846 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:23.027751 (+ 1905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:23.027756 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:23.027759 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:24:23.027780 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:23.027821 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:24:23.027833 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:23.027913 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:23.028202 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 11:24:23.028289 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":289,"threads_started":1}
I20260807 11:24:23.030195 27318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:23.017226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45363 (local address 127.16.197.1:47581)
0807 11:24:23.018267 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:23.018309 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:23.020168 (+ 1859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:23.021069 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:23.021078 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:23.021183 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:23.022014 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:23.022052 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:23.023394 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:23.023399 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:23.025010 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:23.025024 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:23.026027 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:23.026037 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:23.026405 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:23.027329 (+ 924us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 11:24:23.027335 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:23.027400 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 11:24:23.027523 (+ 123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:24:23.027945 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:24:23.027959 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:24:23.027971 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 11:24:23.028187 (+ 216us) client_negotiation.cc:240] Negotiation successful
0807 11:24:23.028668 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":330,"threads_started":1}
I20260807 11:24:23.032029 27315 heartbeater.cc:347] Connected to a master server at 127.16.197.62:45363
I20260807 11:24:23.032369 27315 heartbeater.cc:464] Registering TS with master...
I20260807 11:24:23.033205 27315 heartbeater.cc:511] Master 127.16.197.62:45363 requested a full tablet report, sending...
I20260807 11:24:23.035802 27116 ts_manager.cc:194] Registered new tserver with Master: bba7808bf4ee49d2b505452b3efa977a (127.16.197.1:42773)
I20260807 11:24:23.038146 27116 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.197.1:47581
W20260807 11:24:23.387564 27319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:24:23.387877 27319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:24:23.387972 27319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:24:23.395475 27319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:24:23.395604 27319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:24:23.395713 27319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:24:23.410264 27319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:42831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:45363
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:24:23.411964 27319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:24:23.414346 27319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:24:23.430528 27324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:23.430636 27325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:23.430601 27327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:24:23.431793 27319 server_base.cc:1034] running on GCE node
I20260807 11:24:23.432579 27319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:24:23.433679 27319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:24:23.434904 27319 hybrid_clock.cc:648] HybridClock initialized: now 1786101863434843 us; error 41 us; skew 500 ppm
I20260807 11:24:23.435335 27319 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 11:24:23.438166 27319 webserver.cc:460] Webserver started at http://127.16.197.2:43583/ using document root <none> and password file <none>
I20260807 11:24:23.438902 27319 fs_manager.cc:359] Metadata directory not provided
I20260807 11:24:23.439072 27319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:24:23.439391 27319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:24:23.442490 27319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/data/instance:
uuid: "53e1ada704dc4e339207f0626937f220"
format_stamp: "Formatted at 2026-08-07 11:24:23 on dist-test-slave-2w3w"
server_key: "b746b7cad3b92225c44cdfae6a6d93b8"
server_key_iv: "09956b2926c72546405b876d59f23837"
server_key_version: "encryptionkey@0"
I20260807 11:24:23.443467 27319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance:
uuid: "53e1ada704dc4e339207f0626937f220"
format_stamp: "Formatted at 2026-08-07 11:24:23 on dist-test-slave-2w3w"
server_key: "b746b7cad3b92225c44cdfae6a6d93b8"
server_key_iv: "09956b2926c72546405b876d59f23837"
server_key_version: "encryptionkey@0"
I20260807 11:24:23.449553 27319 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260807 11:24:23.453599 27334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:23.455783 27319 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 11:24:23.456004 27319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "53e1ada704dc4e339207f0626937f220"
format_stamp: "Formatted at 2026-08-07 11:24:23 on dist-test-slave-2w3w"
server_key: "b746b7cad3b92225c44cdfae6a6d93b8"
server_key_iv: "09956b2926c72546405b876d59f23837"
server_key_version: "encryptionkey@0"
I20260807 11:24:23.456269 27319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:24:23.476155 27319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:24:23.477128 27319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:24:23.477484 27319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:24:23.478724 27319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:24:23.480438 27319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:24:23.480573 27319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:23.480690 27319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:24:23.480822 27319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:23.518316 27319 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:45429
I20260807 11:24:23.518383 27451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:45429 every 8 connection(s)
I20260807 11:24:23.519877 27319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:24:23.530095 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 27319
I20260807 11:24:23.530282 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance
I20260807 11:24:23.530872 17172 external_mini_cluster.cc:1428] Setting key 9d6c9de0f993080fee66f5844047b992
I20260807 11:24:23.534407 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:0
--local_ip_for_outbound_sockets=127.16.197.3
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:45363
--builtin_ntp_servers=127.16.197.20:42831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 11:24:23.534598 27317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:23.522899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:59499 (local address 127.16.197.62:45363)
0807 11:24:23.523151 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:23.523179 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:23.524814 (+ 1635us) server_negotiation.cc:407] Connection header received
0807 11:24:23.526625 (+ 1811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:23.526629 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:23.526704 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:23.526805 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:23.528185 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:23.528933 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:23.531340 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:23.532021 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:23.532129 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:23.533759 (+ 1630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:23.533763 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:23.533767 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:24:23.533787 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:23.533820 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:24:23.533832 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:23.533902 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:23.534215 (+ 313us) server_negotiation.cc:299] Negotiation successful
0807 11:24:23.534305 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 11:24:23.536477 27454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:23.523602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45363 (local address 127.16.197.2:59499)
0807 11:24:23.524589 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:23.524635 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:23.526342 (+ 1707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:23.527221 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:23.527230 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:23.527345 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:23.527989 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:23.528009 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:23.529102 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:23.529106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:23.531134 (+ 2028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:23.531160 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:23.532271 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:23.532277 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:23.532494 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:23.533349 (+ 855us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 11:24:23.533353 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:23.533413 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 11:24:23.533524 (+ 111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:24:23.533953 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:24:23.533966 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:24:23.533979 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 11:24:23.534186 (+ 207us) client_negotiation.cc:240] Negotiation successful
0807 11:24:23.534672 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":311,"threads_started":1}
I20260807 11:24:23.538158 27452 heartbeater.cc:347] Connected to a master server at 127.16.197.62:45363
I20260807 11:24:23.538506 27452 heartbeater.cc:464] Registering TS with master...
I20260807 11:24:23.539443 27452 heartbeater.cc:511] Master 127.16.197.62:45363 requested a full tablet report, sending...
I20260807 11:24:23.541083 27116 ts_manager.cc:194] Registered new tserver with Master: 53e1ada704dc4e339207f0626937f220 (127.16.197.2:45429)
I20260807 11:24:23.541945 27116 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.197.2:59499
W20260807 11:24:23.885780 27455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:24:23.886096 27455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:24:23.886250 27455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:24:23.893658 27455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:24:23.893795 27455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:24:23.894016 27455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:24:23.903777 27455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:42831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:45363
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:24:23.905297 27455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:24:23.907377 27455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:24:23.922773 27464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:23.922861 27461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:23.922814 27462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:24:23.923830 27455 server_base.cc:1034] running on GCE node
I20260807 11:24:23.924486 27455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:24:23.925521 27455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:24:23.926743 27455 hybrid_clock.cc:648] HybridClock initialized: now 1786101863926687 us; error 48 us; skew 500 ppm
I20260807 11:24:23.927165 27455 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 11:24:23.929735 27455 webserver.cc:460] Webserver started at http://127.16.197.3:40015/ using document root <none> and password file <none>
I20260807 11:24:23.930546 27455 fs_manager.cc:359] Metadata directory not provided
I20260807 11:24:23.930680 27455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:24:23.931131 27455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:24:23.934481 27455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/data/instance:
uuid: "59b6db5fc82b45109fce8dcc1389ad4b"
format_stamp: "Formatted at 2026-08-07 11:24:23 on dist-test-slave-2w3w"
server_key: "83def8ac9e8a28c6356ca63ee01fb87d"
server_key_iv: "bbf481c8bd88768dddfdd31c12fd367c"
server_key_version: "encryptionkey@0"
I20260807 11:24:23.935468 27455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance:
uuid: "59b6db5fc82b45109fce8dcc1389ad4b"
format_stamp: "Formatted at 2026-08-07 11:24:23 on dist-test-slave-2w3w"
server_key: "83def8ac9e8a28c6356ca63ee01fb87d"
server_key_iv: "bbf481c8bd88768dddfdd31c12fd367c"
server_key_version: "encryptionkey@0"
I20260807 11:24:23.941378 27455 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 11:24:23.945503 27470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:23.947131 27455 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:24:23.947338 27455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "59b6db5fc82b45109fce8dcc1389ad4b"
format_stamp: "Formatted at 2026-08-07 11:24:23 on dist-test-slave-2w3w"
server_key: "83def8ac9e8a28c6356ca63ee01fb87d"
server_key_iv: "bbf481c8bd88768dddfdd31c12fd367c"
server_key_version: "encryptionkey@0"
I20260807 11:24:23.947590 27455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:24:23.964679 27455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:24:23.965593 27455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:24:23.965958 27455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:24:23.967038 27455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:24:23.968557 27455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:24:23.968648 27455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:23.968729 27455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:24:23.968788 27455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:24.004786 27455 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:37207
I20260807 11:24:24.004901 27591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:37207 every 8 connection(s)
I20260807 11:24:24.006227 27455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:24:24.014703 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 27455
I20260807 11:24:24.014925 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance
I20260807 11:24:24.015569 17172 external_mini_cluster.cc:1428] Setting key a9f4d286b4a002ec1f468c14ca359257
I20260807 11:24:24.021275 27317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:24.009035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:48939 (local address 127.16.197.62:45363)
0807 11:24:24.009226 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:24.009231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:24.010690 (+ 1459us) server_negotiation.cc:407] Connection header received
0807 11:24:24.013177 (+ 2487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:24.013182 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:24.013260 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:24.013343 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:24.014940 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:24.016129 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:24.018358 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:24.018827 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:24.019002 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:24.020397 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:24.020402 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:24.020405 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:24:24.020424 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:24.020455 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:24:24.020467 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:24.020572 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:24.020805 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 11:24:24.020922 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 11:24:24.022349 27594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:24.009523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45363 (local address 127.16.197.3:48939)
0807 11:24:24.010497 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:24.010544 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:24.012804 (+ 2260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:24.013632 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:24.013645 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:24.013772 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:24.014644 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:24.014673 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:24.016318 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:24.016349 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:24.018197 (+ 1848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:24.018209 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:24.019108 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:24.019115 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:24.019352 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:24.020052 (+ 700us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 11:24:24.020054 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:24.020096 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 11:24:24.020178 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:24:24.020569 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:24:24.020580 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:24:24.020590 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 11:24:24.020794 (+ 204us) client_negotiation.cc:240] Negotiation successful
0807 11:24:24.021224 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":307,"threads_started":1}
I20260807 11:24:24.023780 27592 heartbeater.cc:347] Connected to a master server at 127.16.197.62:45363
I20260807 11:24:24.024116 27592 heartbeater.cc:464] Registering TS with master...
I20260807 11:24:24.024982 27592 heartbeater.cc:511] Master 127.16.197.62:45363 requested a full tablet report, sending...
I20260807 11:24:24.026568 27116 ts_manager.cc:194] Registered new tserver with Master: 59b6db5fc82b45109fce8dcc1389ad4b (127.16.197.3:37207)
I20260807 11:24:24.027482 27116 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.197.3:48939
I20260807 11:24:24.031725 17172 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 11:24:24.038821 27317 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:24.036926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34370 (local address 127.16.197.62:45363)
0807 11:24:24.037193 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:24.037202 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:24.037674 (+ 472us) server_negotiation.cc:407] Connection header received
0807 11:24:24.037942 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:24.037949 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:24.038023 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:24.038143 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:24.038596 (+ 453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34370: BlockingRecv error: recv got EOF from 127.0.0.1:34370 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 11:24:24.039857 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 27180
I20260807 11:24:24.042492 27315 heartbeater.cc:503] Master 127.16.197.62:45363 was elected leader, sending a full tablet report...
I20260807 11:24:24.048944 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:24.040906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45176 (local address 127.16.197.1:42773)
0807 11:24:24.041544 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:24.041552 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:24.041578 (+ 26us) server_negotiation.cc:407] Connection header received
0807 11:24:24.041641 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:24.041648 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:24.041876 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:24.042021 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:24.042499 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:24.043974 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:24.045434 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:24.046021 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:24.046132 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:24.047388 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:24.047397 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:24.047412 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:24:24.048050 (+ 638us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:24.048101 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:24:24.048122 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:24.048267 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:24.048433 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 11:24:24.048578 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":336,"threads_started":1}
I20260807 11:24:24.050305 27309 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:24:24.136121 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 27180
I20260807 11:24:24.155839 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 27319
I20260807 11:24:24.164569 27604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:24.156920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49188 (local address 127.16.197.2:45429)
0807 11:24:24.157505 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:24.157512 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:24.157540 (+ 28us) server_negotiation.cc:407] Connection header received
0807 11:24:24.157602 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:24.157609 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:24.157833 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:24.157969 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:24.158378 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:24.159502 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:24.160966 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:24.161696 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:24.161860 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:24.163023 (+ 1163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:24.163038 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:24.163061 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:24:24.163730 (+ 669us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:24.163796 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:24:24.163818 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:24.163918 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:24.164110 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 11:24:24.164253 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":292,"threads_started":1}
I20260807 11:24:24.165961 27447 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:24:24.253726 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 27319
I20260807 11:24:24.272558 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 27455
I20260807 11:24:24.281536 27606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:24.273597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46728 (local address 127.16.197.3:37207)
0807 11:24:24.274359 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:24.274367 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:24.274401 (+ 34us) server_negotiation.cc:407] Connection header received
0807 11:24:24.274468 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:24.274475 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:24.274701 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:24.274931 (+ 230us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:24.275472 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:24.276543 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:24.277996 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:24.278592 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:24.278747 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:24.279958 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:24.279966 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:24.279982 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:24:24.280598 (+ 616us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:24.280656 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:24:24.280676 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:24.280813 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:24.280984 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 11:24:24.281152 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":426,"threads_started":1}
I20260807 11:24:24.282910 27587 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:24:24.369637 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 27455
I20260807 11:24:24.388335 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 27081
I20260807 11:24:24.389712 27148 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:24:24.496867 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 27081
2026-08-07T11:24:24Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2675 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T11:24:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:24:24Z Disabled control of system clock
I20260807 11:24:24.579962 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:45901
--webserver_interface=127.16.197.62
--webserver_port=0
--builtin_ntp_servers=127.16.197.20:33037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:45901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 11:24:24.935031 27613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:24:24.935492 27613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:24:24.935649 27613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:24:24.943461 27613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:24:24.943594 27613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:24:24.943701 27613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:24:24.943792 27613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:24:24.943853 27613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:24:24.953799 27613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:33037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:45901
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:45901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:24:24.955374 27613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:24:24.957342 27613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:24:24.970721 27618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:24.970647 27619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:24.970698 27621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:24:24.971666 27613 server_base.cc:1034] running on GCE node
I20260807 11:24:24.972579 27613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:24:24.974164 27613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:24:24.975381 27613 hybrid_clock.cc:648] HybridClock initialized: now 1786101864975342 us; error 38 us; skew 500 ppm
I20260807 11:24:24.975775 27613 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 11:24:24.978317 27613 webserver.cc:460] Webserver started at http://127.16.197.62:35665/ using document root <none> and password file <none>
I20260807 11:24:24.979288 27613 fs_manager.cc:359] Metadata directory not provided
I20260807 11:24:24.979429 27613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:24:24.979820 27613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:24:24.983340 27613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "a1c93d46fa1241e38cf912a8eac0c17f"
format_stamp: "Formatted at 2026-08-07 11:24:24 on dist-test-slave-2w3w"
server_key: "f2981706e8221239adf94018d75e40b1"
server_key_iv: "c013a6b089e5fa4b22fe8d9b79999ebe"
server_key_version: "encryptionkey@0"
I20260807 11:24:24.984310 27613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "a1c93d46fa1241e38cf912a8eac0c17f"
format_stamp: "Formatted at 2026-08-07 11:24:24 on dist-test-slave-2w3w"
server_key: "f2981706e8221239adf94018d75e40b1"
server_key_iv: "c013a6b089e5fa4b22fe8d9b79999ebe"
server_key_version: "encryptionkey@0"
I20260807 11:24:24.990708 27613 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260807 11:24:24.995299 27627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:24.997052 27613 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 11:24:24.997318 27613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "a1c93d46fa1241e38cf912a8eac0c17f"
format_stamp: "Formatted at 2026-08-07 11:24:24 on dist-test-slave-2w3w"
server_key: "f2981706e8221239adf94018d75e40b1"
server_key_iv: "c013a6b089e5fa4b22fe8d9b79999ebe"
server_key_version: "encryptionkey@0"
I20260807 11:24:24.997611 27613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:24:25.004642 27613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:24:25.005659 27613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:24:25.006032 27613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:24:25.027280 27613 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:45901
I20260807 11:24:25.027349 27681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:45901 every 8 connection(s)
I20260807 11:24:25.028854 27613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:24:25.033015 27682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:24:25.037767 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 27613
I20260807 11:24:25.038040 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:24:25.038725 17172 external_mini_cluster.cc:1428] Setting key d8b23d2cc208381387d36a32fd746a9b
I20260807 11:24:25.045977 27682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1c93d46fa1241e38cf912a8eac0c17f: Bootstrap starting.
I20260807 11:24:25.049832 27685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:25.041243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42702 (local address 127.16.197.62:45901)
0807 11:24:25.042449 (+ 1206us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:25.042472 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:25.042520 (+ 48us) server_negotiation.cc:407] Connection header received
0807 11:24:25.044494 (+ 1974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:25.044536 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:25.045318 (+ 782us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:25.045908 (+ 590us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:25.047430 (+ 1522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:25.047438 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:25.047458 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:24:25.047481 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:25.047597 (+ 116us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:24:25.047623 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:25.047888 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:25.048102 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 11:24:25.048389 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":456,"threads_started":1}
I20260807 11:24:25.052340 27682 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1c93d46fa1241e38cf912a8eac0c17f: Neither blocks nor log segments found. Creating new log.
I20260807 11:24:25.053805 27682 log.cc:824] T 00000000000000000000000000000000 P a1c93d46fa1241e38cf912a8eac0c17f: Log is configured to *not* fsync() on all Append() calls
I20260807 11:24:25.057210 27682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1c93d46fa1241e38cf912a8eac0c17f: No bootstrap required, opened a new log
I20260807 11:24:25.065165 27682 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1c93d46fa1241e38cf912a8eac0c17f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c93d46fa1241e38cf912a8eac0c17f" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45901 } }
I20260807 11:24:25.065586 27682 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1c93d46fa1241e38cf912a8eac0c17f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:24:25.065730 27682 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1c93d46fa1241e38cf912a8eac0c17f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1c93d46fa1241e38cf912a8eac0c17f, State: Initialized, Role: FOLLOWER
I20260807 11:24:25.066373 27682 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1c93d46fa1241e38cf912a8eac0c17f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c93d46fa1241e38cf912a8eac0c17f" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45901 } }
I20260807 11:24:25.066596 27682 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1c93d46fa1241e38cf912a8eac0c17f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:24:25.066792 27682 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1c93d46fa1241e38cf912a8eac0c17f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:24:25.067024 27682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1c93d46fa1241e38cf912a8eac0c17f [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:24:25.069386 27682 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1c93d46fa1241e38cf912a8eac0c17f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c93d46fa1241e38cf912a8eac0c17f" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45901 } }
I20260807 11:24:25.069972 27682 leader_election.cc:302] T 00000000000000000000000000000000 P a1c93d46fa1241e38cf912a8eac0c17f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1c93d46fa1241e38cf912a8eac0c17f; no voters:
I20260807 11:24:25.070793 27682 leader_election.cc:288] T 00000000000000000000000000000000 P a1c93d46fa1241e38cf912a8eac0c17f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:24:25.070955 27687 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1c93d46fa1241e38cf912a8eac0c17f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:24:25.071734 27687 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1c93d46fa1241e38cf912a8eac0c17f [term 1 LEADER]: Becoming Leader. State: Replica: a1c93d46fa1241e38cf912a8eac0c17f, State: Running, Role: LEADER
I20260807 11:24:25.072365 27687 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1c93d46fa1241e38cf912a8eac0c17f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c93d46fa1241e38cf912a8eac0c17f" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45901 } }
I20260807 11:24:25.073444 27682 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1c93d46fa1241e38cf912a8eac0c17f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:24:25.075856 27688 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1c93d46fa1241e38cf912a8eac0c17f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1c93d46fa1241e38cf912a8eac0c17f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c93d46fa1241e38cf912a8eac0c17f" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45901 } } }
I20260807 11:24:25.075937 27689 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1c93d46fa1241e38cf912a8eac0c17f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1c93d46fa1241e38cf912a8eac0c17f. Latest consensus state: current_term: 1 leader_uuid: "a1c93d46fa1241e38cf912a8eac0c17f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c93d46fa1241e38cf912a8eac0c17f" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45901 } } }
I20260807 11:24:25.076292 27688 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1c93d46fa1241e38cf912a8eac0c17f [sys.catalog]: This master's current role is: LEADER
I20260807 11:24:25.076421 27689 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1c93d46fa1241e38cf912a8eac0c17f [sys.catalog]: This master's current role is: LEADER
I20260807 11:24:25.079026 27693 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:24:25.084618 27693 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:24:25.094110 27693 catalog_manager.cc:1347] Generated new cluster ID: aec6efa8020e47219591c570e19c0164
I20260807 11:24:25.094270 27693 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:24:25.122030 27693 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:24:25.122552 27693 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:24:25.129460 27693 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1c93d46fa1241e38cf912a8eac0c17f: Generated new TSK 0
I20260807 11:24:25.130769 27693 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:24:25.136749 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:0
--local_ip_for_outbound_sockets=127.16.197.1
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:45901
--builtin_ntp_servers=127.16.197.20:33037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 11:24:25.555128 27708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:24:25.555510 27708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:24:25.555675 27708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:24:25.563328 27708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:24:25.563489 27708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:24:25.563673 27708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:24:25.573941 27708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:33037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:45901
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:24:25.575690 27708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:24:25.577960 27708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:24:25.593509 27713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:25.593483 27716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:25.593536 27714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:24:25.594434 27708 server_base.cc:1034] running on GCE node
I20260807 11:24:25.595224 27708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:24:25.596382 27708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:24:25.597644 27708 hybrid_clock.cc:648] HybridClock initialized: now 1786101865597601 us; error 46 us; skew 500 ppm
I20260807 11:24:25.598114 27708 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 11:24:25.600811 27708 webserver.cc:460] Webserver started at http://127.16.197.1:37025/ using document root <none> and password file <none>
I20260807 11:24:25.601727 27708 fs_manager.cc:359] Metadata directory not provided
I20260807 11:24:25.601873 27708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:24:25.602226 27708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:24:25.605611 27708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/data/instance:
uuid: "c35a01ae2c8e4f2d9b4c610864377b68"
format_stamp: "Formatted at 2026-08-07 11:24:25 on dist-test-slave-2w3w"
server_key: "1196e8611db9582e93ac663558545e9d"
server_key_iv: "281dbd57fdc09ccba6117021adf0453f"
server_key_version: "encryptionkey@0"
I20260807 11:24:25.606562 27708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance:
uuid: "c35a01ae2c8e4f2d9b4c610864377b68"
format_stamp: "Formatted at 2026-08-07 11:24:25 on dist-test-slave-2w3w"
server_key: "1196e8611db9582e93ac663558545e9d"
server_key_iv: "281dbd57fdc09ccba6117021adf0453f"
server_key_version: "encryptionkey@0"
I20260807 11:24:25.612600 27708 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.005s
I20260807 11:24:25.616531 27722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:25.618140 27708 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 11:24:25.618372 27708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "c35a01ae2c8e4f2d9b4c610864377b68"
format_stamp: "Formatted at 2026-08-07 11:24:25 on dist-test-slave-2w3w"
server_key: "1196e8611db9582e93ac663558545e9d"
server_key_iv: "281dbd57fdc09ccba6117021adf0453f"
server_key_version: "encryptionkey@0"
I20260807 11:24:25.618633 27708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:24:25.625124 27708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:24:25.626035 27708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:24:25.626351 27708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:24:25.627496 27708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:24:25.629207 27708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:24:25.629360 27708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:25.629493 27708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:24:25.629599 27708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:25.665903 27708 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:36219
I20260807 11:24:25.666019 27839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:36219 every 8 connection(s)
I20260807 11:24:25.667393 27708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:24:25.672772 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 27708
I20260807 11:24:25.673046 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance
I20260807 11:24:25.673862 17172 external_mini_cluster.cc:1428] Setting key 3bbcc24b37937204b9864c1f727e74b7
I20260807 11:24:25.676967 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:0
--local_ip_for_outbound_sockets=127.16.197.2
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:45901
--builtin_ntp_servers=127.16.197.20:33037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 11:24:25.677356 27842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:25.670198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:44297 (local address 127.16.197.62:45901)
0807 11:24:25.670677 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:25.670682 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:25.672019 (+ 1337us) server_negotiation.cc:407] Connection header received
0807 11:24:25.674770 (+ 2751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:25.674776 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:25.674871 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:25.675026 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:25.676509 (+ 1483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:25.676515 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:25.676519 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:24:25.676539 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:25.676594 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:24:25.676607 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:25.676748 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:25.676955 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 11:24:25.677030 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":234,"threads_started":1}
I20260807 11:24:25.678514 27843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:25.670860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45901 (local address 127.16.197.1:44297)
0807 11:24:25.671818 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:25.671863 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:25.674318 (+ 2455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:25.675301 (+ 983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:25.675310 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:25.675419 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:25.676185 (+ 766us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 11:24:25.676188 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:25.676228 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 11:24:25.676328 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:24:25.676712 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:24:25.676727 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:24:25.676735 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 11:24:25.677053 (+ 318us) client_negotiation.cc:240] Negotiation successful
0807 11:24:25.677356 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":275,"threads_started":1}
I20260807 11:24:25.680289 27840 heartbeater.cc:347] Connected to a master server at 127.16.197.62:45901
I20260807 11:24:25.680671 27840 heartbeater.cc:464] Registering TS with master...
I20260807 11:24:25.681413 27840 heartbeater.cc:511] Master 127.16.197.62:45901 requested a full tablet report, sending...
I20260807 11:24:25.684041 27644 ts_manager.cc:194] Registered new tserver with Master: c35a01ae2c8e4f2d9b4c610864377b68 (127.16.197.1:36219)
W20260807 11:24:26.040314 27844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:24:26.040726 27844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:24:26.040868 27844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:24:26.052330 27844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:24:26.052454 27844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:24:26.052572 27844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:24:26.062237 27844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:33037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:45901
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:24:26.063692 27844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:24:26.065563 27844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:24:26.080102 27849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:26.080085 27852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:26.080076 27850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:24:26.081216 27844 server_base.cc:1034] running on GCE node
I20260807 11:24:26.081923 27844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:24:26.083236 27844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:24:26.084494 27844 hybrid_clock.cc:648] HybridClock initialized: now 1786101866084437 us; error 81 us; skew 500 ppm
I20260807 11:24:26.084857 27844 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 11:24:26.087661 27844 webserver.cc:460] Webserver started at http://127.16.197.2:45285/ using document root <none> and password file <none>
I20260807 11:24:26.088466 27844 fs_manager.cc:359] Metadata directory not provided
I20260807 11:24:26.088613 27844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:24:26.088963 27844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:24:26.092207 27844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/data/instance:
uuid: "bb6ef1d313fb46c582cfee5510a89025"
format_stamp: "Formatted at 2026-08-07 11:24:26 on dist-test-slave-2w3w"
server_key: "6ddfc0bdd9064aa817da9dd9d8e9eab5"
server_key_iv: "5eb8b79a70914f55e6bc17215cb121f7"
server_key_version: "encryptionkey@0"
I20260807 11:24:26.093142 27844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance:
uuid: "bb6ef1d313fb46c582cfee5510a89025"
format_stamp: "Formatted at 2026-08-07 11:24:26 on dist-test-slave-2w3w"
server_key: "6ddfc0bdd9064aa817da9dd9d8e9eab5"
server_key_iv: "5eb8b79a70914f55e6bc17215cb121f7"
server_key_version: "encryptionkey@0"
I20260807 11:24:26.099243 27844 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260807 11:24:26.103623 27858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:26.105329 27844 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260807 11:24:26.105552 27844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "bb6ef1d313fb46c582cfee5510a89025"
format_stamp: "Formatted at 2026-08-07 11:24:26 on dist-test-slave-2w3w"
server_key: "6ddfc0bdd9064aa817da9dd9d8e9eab5"
server_key_iv: "5eb8b79a70914f55e6bc17215cb121f7"
server_key_version: "encryptionkey@0"
I20260807 11:24:26.105815 27844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:24:26.112252 27844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:24:26.113178 27844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:24:26.113480 27844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:24:26.114538 27844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:24:26.116263 27844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:24:26.116356 27844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:26.116524 27844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:24:26.116611 27844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:26.153860 27844 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:40413
I20260807 11:24:26.153988 27978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:40413 every 8 connection(s)
I20260807 11:24:26.155429 27844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:24:26.156863 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 27844
I20260807 11:24:26.157078 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance
I20260807 11:24:26.157557 17172 external_mini_cluster.cc:1428] Setting key 47f5ea97f32c60823df0b7f3f2c3c09f
I20260807 11:24:26.161334 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:0
--local_ip_for_outbound_sockets=127.16.197.3
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:45901
--builtin_ntp_servers=127.16.197.20:33037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 11:24:26.166502 27842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:26.158536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:59215 (local address 127.16.197.62:45901)
0807 11:24:26.158760 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:26.158767 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:26.160845 (+ 2078us) server_negotiation.cc:407] Connection header received
0807 11:24:26.163833 (+ 2988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:26.163838 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:26.163902 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:26.163999 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:26.165671 (+ 1672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:26.165675 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:26.165678 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:24:26.165698 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:26.165732 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:24:26.165746 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:26.165864 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:26.166180 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 11:24:26.166248 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 11:24:26.167963 27981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:26.159237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45901 (local address 127.16.197.2:59215)
0807 11:24:26.160628 (+ 1391us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:26.160691 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:26.163450 (+ 2759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:26.164334 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:26.164349 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:26.164491 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:26.165277 (+ 786us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 11:24:26.165282 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:26.165340 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 11:24:26.165469 (+ 129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:24:26.165852 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:24:26.165865 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:24:26.165877 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 11:24:26.166202 (+ 325us) client_negotiation.cc:240] Negotiation successful
0807 11:24:26.166512 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"thread_start_us":503,"threads_started":1}
I20260807 11:24:26.169672 27979 heartbeater.cc:347] Connected to a master server at 127.16.197.62:45901
I20260807 11:24:26.170029 27979 heartbeater.cc:464] Registering TS with master...
I20260807 11:24:26.170671 27979 heartbeater.cc:511] Master 127.16.197.62:45901 requested a full tablet report, sending...
I20260807 11:24:26.172315 27644 ts_manager.cc:194] Registered new tserver with Master: bb6ef1d313fb46c582cfee5510a89025 (127.16.197.2:40413)
W20260807 11:24:26.509965 27983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:24:26.510354 27983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:24:26.510552 27983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:24:26.518323 27983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:24:26.518504 27983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:24:26.518687 27983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:24:26.528478 27983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:33037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:45901
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:24:26.529902 27983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:24:26.531970 27983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:24:26.545595 27988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:26.545537 27989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:26.545567 27991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:24:26.546422 27983 server_base.cc:1034] running on GCE node
I20260807 11:24:26.547221 27983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:24:26.548249 27983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:24:26.549504 27983 hybrid_clock.cc:648] HybridClock initialized: now 1786101866549466 us; error 59 us; skew 500 ppm
I20260807 11:24:26.549914 27983 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 11:24:26.552455 27983 webserver.cc:460] Webserver started at http://127.16.197.3:36407/ using document root <none> and password file <none>
I20260807 11:24:26.553224 27983 fs_manager.cc:359] Metadata directory not provided
I20260807 11:24:26.553370 27983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:24:26.553725 27983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:24:26.556797 27983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/data/instance:
uuid: "ab65b6acfb6742d4bcb6df7b34816af5"
format_stamp: "Formatted at 2026-08-07 11:24:26 on dist-test-slave-2w3w"
server_key: "86bf7be4d4100946e211bae6abef262e"
server_key_iv: "a0e201920df488fe1bf5a5e1e8f69934"
server_key_version: "encryptionkey@0"
I20260807 11:24:26.557667 27983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance:
uuid: "ab65b6acfb6742d4bcb6df7b34816af5"
format_stamp: "Formatted at 2026-08-07 11:24:26 on dist-test-slave-2w3w"
server_key: "86bf7be4d4100946e211bae6abef262e"
server_key_iv: "a0e201920df488fe1bf5a5e1e8f69934"
server_key_version: "encryptionkey@0"
I20260807 11:24:26.563326 27983 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 11:24:26.566884 27997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:26.568446 27983 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 11:24:26.568588 27983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "ab65b6acfb6742d4bcb6df7b34816af5"
format_stamp: "Formatted at 2026-08-07 11:24:26 on dist-test-slave-2w3w"
server_key: "86bf7be4d4100946e211bae6abef262e"
server_key_iv: "a0e201920df488fe1bf5a5e1e8f69934"
server_key_version: "encryptionkey@0"
I20260807 11:24:26.568778 27983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:24:26.574532 27983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:24:26.575495 27983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:24:26.575730 27983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:24:26.576635 27983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:24:26.578035 27983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:24:26.578122 27983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:26.578207 27983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:24:26.578261 27983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:26.612177 27983 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:34135
I20260807 11:24:26.612236 28114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:34135 every 8 connection(s)
I20260807 11:24:26.613603 27983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:24:26.622201 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 27983
I20260807 11:24:26.622434 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance
I20260807 11:24:26.623095 17172 external_mini_cluster.cc:1428] Setting key ac9551cefe3a236cc83b90cc81c50c04
I20260807 11:24:26.623275 27842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:26.616691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:52861 (local address 127.16.197.62:45901)
0807 11:24:26.616945 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:26.616952 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:26.618386 (+ 1434us) server_negotiation.cc:407] Connection header received
0807 11:24:26.620272 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:26.620277 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:26.620343 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:26.620445 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:26.622115 (+ 1670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:26.622120 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:26.622124 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:24:26.622145 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:26.622182 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:24:26.622196 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:26.622448 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:26.622818 (+ 370us) server_negotiation.cc:299] Negotiation successful
0807 11:24:26.622905 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 11:24:26.624475 28117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:26.617145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45901 (local address 127.16.197.3:52861)
0807 11:24:26.618202 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:26.618242 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:26.619968 (+ 1726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:26.620749 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:26.620758 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:26.620861 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:26.621617 (+ 756us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 11:24:26.621620 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:26.621671 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 11:24:26.621843 (+ 172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:24:26.622424 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:24:26.622438 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:24:26.622448 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 11:24:26.622837 (+ 389us) client_negotiation.cc:240] Negotiation successful
0807 11:24:26.623201 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":344,"threads_started":1}
I20260807 11:24:26.626109 28115 heartbeater.cc:347] Connected to a master server at 127.16.197.62:45901
I20260807 11:24:26.626466 28115 heartbeater.cc:464] Registering TS with master...
I20260807 11:24:26.627260 28115 heartbeater.cc:511] Master 127.16.197.62:45901 requested a full tablet report, sending...
I20260807 11:24:26.628860 27643 ts_manager.cc:194] Registered new tserver with Master: ab65b6acfb6742d4bcb6df7b34816af5 (127.16.197.3:34135)
I20260807 11:24:26.639631 17172 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 11:24:26.646265 27842 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:24:26.644632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42710 (local address 127.16.197.62:45901)
0807 11:24:26.644834 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:26.644840 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:26.645355 (+ 515us) server_negotiation.cc:407] Connection header received
0807 11:24:26.645506 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:26.645510 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:26.645567 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:26.645662 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:26.646045 (+ 383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42710: BlockingRecv error: recv got EOF from 127.0.0.1:42710 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 11:24:26.646941 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 27708
I20260807 11:24:26.651363 28124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:26.648444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55116 (local address 127.16.197.1:36219)
0807 11:24:26.649062 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:26.649070 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:26.649099 (+ 29us) server_negotiation.cc:407] Connection header received
0807 11:24:26.649159 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:26.649166 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:26.649389 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:26.649523 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:26.650451 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:26.650457 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:26.650468 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:24:26.650486 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:26.650525 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:24:26.650539 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:26.650648 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:26.650857 (+ 209us) server_negotiation.cc:299] Negotiation successful
0807 11:24:26.651060 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":302,"threads_started":1}
I20260807 11:24:26.652603 27835 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:24:26.733603 27840 heartbeater.cc:503] Master 127.16.197.62:45901 was elected leader, sending a full tablet report...
I20260807 11:24:26.735716 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 27708
I20260807 11:24:26.754341 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 27844
I20260807 11:24:26.758512 28126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:26.755431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45544 (local address 127.16.197.2:40413)
0807 11:24:26.756016 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:26.756024 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:26.756052 (+ 28us) server_negotiation.cc:407] Connection header received
0807 11:24:26.756117 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:26.756124 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:26.756365 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:26.756555 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:26.757694 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:26.757701 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:26.757713 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:24:26.757732 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:26.757781 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:24:26.757797 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:26.757959 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:26.758114 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 11:24:26.758226 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":291,"threads_started":1}
I20260807 11:24:26.760087 27974 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:24:26.843961 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 27844
I20260807 11:24:26.862687 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 27983
I20260807 11:24:26.867359 28128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:26.863910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36212 (local address 127.16.197.3:34135)
0807 11:24:26.864621 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:26.864629 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:26.864661 (+ 32us) server_negotiation.cc:407] Connection header received
0807 11:24:26.864726 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:26.864734 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:26.865025 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:26.865221 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:26.866499 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:26.866506 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:26.866519 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:24:26.866535 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:26.866574 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:24:26.866589 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:26.866806 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:26.866891 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 11:24:26.867043 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":353,"threads_started":1}
I20260807 11:24:26.868821 28109 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:24:26.953884 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 27983
I20260807 11:24:26.971889 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 27613
I20260807 11:24:26.973156 27676 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:24:27.082909 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 27613
2026-08-07T11:24:27Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2586 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:24:27 dist-test-slave-2w3w krb5kdc[28133](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:24:27 dist-test-slave-2w3w krb5kdc[28133](info): set up 2 sockets
Aug 07 11:24:27 dist-test-slave-2w3w krb5kdc[28133](info): commencing operation
krb5kdc: starting...
W20260807 11:24:28.241546 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.005s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:24:28 dist-test-slave-2w3w krb5kdc[28133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:24:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:24:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:24:28.519505 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:41365
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:34589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:41365
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:24:28.863320 28147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:24:28.863721 28147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:24:28.863912 28147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:24:28.871621 28147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:24:28.871802 28147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:24:28.871922 28147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:24:28.872022 28147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:24:28.872112 28147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:24:28.882616 28147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:34589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:41365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:41365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:24:28.884219 28147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:24:28.885650 28147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:24:28.899364 28152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:28.899376 28153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:28.899952 28155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:24:28.900668 28147 server_base.cc:1034] running on GCE node
I20260807 11:24:28.901714 28147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:24:28.903491 28147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:24:28.904742 28147 hybrid_clock.cc:648] HybridClock initialized: now 1786101868904678 us; error 54 us; skew 500 ppm
I20260807 11:24:28.905174 28147 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:24:28 dist-test-slave-2w3w krb5kdc[28133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101868, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:24:28.913826 28147 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:24:28.915947 28147 webserver.cc:460] Webserver started at http://127.16.197.62:41539/ using document root <none> and password file <none>
I20260807 11:24:28.916754 28147 fs_manager.cc:359] Metadata directory not provided
I20260807 11:24:28.916903 28147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:24:28.917256 28147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:24:28.920629 28147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "e2844a6c29344025abf64e1c08512f53"
format_stamp: "Formatted at 2026-08-07 11:24:28 on dist-test-slave-2w3w"
server_key: "87f81b86220f9bad959a2e8c56038dd4"
server_key_iv: "b7bf8dd9e0a2aaf5bbe291df72d0fb57"
server_key_version: "encryptionkey@0"
I20260807 11:24:28.921501 28147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "e2844a6c29344025abf64e1c08512f53"
format_stamp: "Formatted at 2026-08-07 11:24:28 on dist-test-slave-2w3w"
server_key: "87f81b86220f9bad959a2e8c56038dd4"
server_key_iv: "b7bf8dd9e0a2aaf5bbe291df72d0fb57"
server_key_version: "encryptionkey@0"
I20260807 11:24:28.927557 28147 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 11:24:28.931557 28163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:28.933248 28147 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 11:24:28.933497 28147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "e2844a6c29344025abf64e1c08512f53"
format_stamp: "Formatted at 2026-08-07 11:24:28 on dist-test-slave-2w3w"
server_key: "87f81b86220f9bad959a2e8c56038dd4"
server_key_iv: "b7bf8dd9e0a2aaf5bbe291df72d0fb57"
server_key_version: "encryptionkey@0"
I20260807 11:24:28.933753 28147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:24:28.952855 28147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:24:28.957798 28147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:24:28.958143 28147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:24:28.978416 28147 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:41365
I20260807 11:24:28.978449 28216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:41365 every 8 connection(s)
I20260807 11:24:28.979974 28147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:24:28.983909 28217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:24:28.987358 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 28147
I20260807 11:24:28.987637 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:24:28.988421 17172 external_mini_cluster.cc:1428] Setting key add231ac0825b187bfb004a67c29a7fe
I20260807 11:24:28.998600 28217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2844a6c29344025abf64e1c08512f53: Bootstrap starting.
I20260807 11:24:29.002687 28217 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2844a6c29344025abf64e1c08512f53: Neither blocks nor log segments found. Creating new log.
I20260807 11:24:29.003859 28217 log.cc:824] T 00000000000000000000000000000000 P e2844a6c29344025abf64e1c08512f53: Log is configured to *not* fsync() on all Append() calls
Aug 07 11:24:29 dist-test-slave-2w3w krb5kdc[28133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:24:29.007552 28217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2844a6c29344025abf64e1c08512f53: No bootstrap required, opened a new log
I20260807 11:24:29.014675 28220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:28.990789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48042 (local address 127.16.197.62:41365)
0807 11:24:28.991777 (+ 988us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:28.991794 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:28.991851 (+ 57us) server_negotiation.cc:407] Connection header received
0807 11:24:28.993910 (+ 2059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:28.993954 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:28.994592 (+ 638us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:28.995189 (+ 597us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:28.996688 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:28.998751 (+ 2063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:29.000287 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:29.001439 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:29.001653 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:29.006393 (+ 4740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:29.006403 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:29.006419 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:24:29.006469 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:29.010003 (+ 3534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:29.011048 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:29.011059 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:29.011066 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:29.011245 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:29.011709 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:29.011712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:29.011714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:29.012287 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:29.012549 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:29.013058 (+ 509us) server_negotiation.cc:299] Negotiation successful
0807 11:24:29.013501 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":718,"thread_start_us":406,"threads_started":1}
I20260807 11:24:29.016835 28217 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2844a6c29344025abf64e1c08512f53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2844a6c29344025abf64e1c08512f53" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 41365 } }
I20260807 11:24:29.017246 28217 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2844a6c29344025abf64e1c08512f53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:24:29.017359 28217 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2844a6c29344025abf64e1c08512f53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2844a6c29344025abf64e1c08512f53, State: Initialized, Role: FOLLOWER
I20260807 11:24:29.018056 28217 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2844a6c29344025abf64e1c08512f53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2844a6c29344025abf64e1c08512f53" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 41365 } }
I20260807 11:24:29.018301 28217 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2844a6c29344025abf64e1c08512f53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:24:29.018438 28217 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2844a6c29344025abf64e1c08512f53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:24:29.018621 28217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2844a6c29344025abf64e1c08512f53 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:24:29.021101 28217 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2844a6c29344025abf64e1c08512f53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2844a6c29344025abf64e1c08512f53" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 41365 } }
I20260807 11:24:29.021690 28217 leader_election.cc:302] T 00000000000000000000000000000000 P e2844a6c29344025abf64e1c08512f53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2844a6c29344025abf64e1c08512f53; no voters:
I20260807 11:24:29.022327 28217 leader_election.cc:288] T 00000000000000000000000000000000 P e2844a6c29344025abf64e1c08512f53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:24:29.022497 28222 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2844a6c29344025abf64e1c08512f53 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:24:29.023283 28222 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2844a6c29344025abf64e1c08512f53 [term 1 LEADER]: Becoming Leader. State: Replica: e2844a6c29344025abf64e1c08512f53, State: Running, Role: LEADER
I20260807 11:24:29.023892 28222 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2844a6c29344025abf64e1c08512f53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2844a6c29344025abf64e1c08512f53" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 41365 } }
I20260807 11:24:29.024700 28217 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2844a6c29344025abf64e1c08512f53 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:24:29.027007 28224 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2844a6c29344025abf64e1c08512f53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2844a6c29344025abf64e1c08512f53. Latest consensus state: current_term: 1 leader_uuid: "e2844a6c29344025abf64e1c08512f53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2844a6c29344025abf64e1c08512f53" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 41365 } } }
I20260807 11:24:29.027513 28224 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2844a6c29344025abf64e1c08512f53 [sys.catalog]: This master's current role is: LEADER
I20260807 11:24:29.028097 28223 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2844a6c29344025abf64e1c08512f53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2844a6c29344025abf64e1c08512f53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2844a6c29344025abf64e1c08512f53" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 41365 } } }
I20260807 11:24:29.028415 28223 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2844a6c29344025abf64e1c08512f53 [sys.catalog]: This master's current role is: LEADER
W20260807 11:24:29.035640 28239 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e2844a6c29344025abf64e1c08512f53: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 11:24:29.035768 28239 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 11:24:29.035905 28231 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:24:29.038203 28231 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:24:29.048629 28231 catalog_manager.cc:1347] Generated new cluster ID: eab76f94f17242ec9745706b69e7d692
I20260807 11:24:29.048818 28231 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:24:29.089982 28231 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:24:29.092032 28231 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:24:29.106889 28231 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2844a6c29344025abf64e1c08512f53: Generated new TSK 0
I20260807 11:24:29.108144 28231 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.197.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
I20260807 11:24:29.179265 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:0
--local_ip_for_outbound_sockets=127.16.197.1
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:41365
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:34589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:24:29.526877 28244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:24:29.527225 28244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:24:29.527328 28244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:24:29.534940 28244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:24:29.535120 28244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:24:29.535236 28244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:24:29.545297 28244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:34589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:41365
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:24:29.546625 28244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:24:29.547991 28244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:24:29.561553 28250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:29.561488 28252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:29.561590 28249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:24:29.563100 28244 server_base.cc:1034] running on GCE node
I20260807 11:24:29.563750 28244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:24:29.564751 28244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:24:29.566035 28244 hybrid_clock.cc:648] HybridClock initialized: now 1786101869565975 us; error 46 us; skew 500 ppm
I20260807 11:24:29.566391 28244 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:24:29 dist-test-slave-2w3w krb5kdc[28133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101869, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:24:29.569720 28244 init.cc:377] Logged in from keytab as kudu/127.16.197.1@KRBTEST.COM (short username kudu)
I20260807 11:24:29.571369 28244 webserver.cc:460] Webserver started at http://127.16.197.1:42809/ using document root <none> and password file <none>
I20260807 11:24:29.572108 28244 fs_manager.cc:359] Metadata directory not provided
I20260807 11:24:29.572256 28244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:24:29.572571 28244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:24:29.575752 28244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/data/instance:
uuid: "b371b50bbc0045d382934fdde294d11d"
format_stamp: "Formatted at 2026-08-07 11:24:29 on dist-test-slave-2w3w"
server_key: "8e746bda55ebf70951334037ec2dbbd6"
server_key_iv: "ebcd906bb1dfb59e975425cfa265fdcb"
server_key_version: "encryptionkey@0"
I20260807 11:24:29.576566 28244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance:
uuid: "b371b50bbc0045d382934fdde294d11d"
format_stamp: "Formatted at 2026-08-07 11:24:29 on dist-test-slave-2w3w"
server_key: "8e746bda55ebf70951334037ec2dbbd6"
server_key_iv: "ebcd906bb1dfb59e975425cfa265fdcb"
server_key_version: "encryptionkey@0"
I20260807 11:24:29.582108 28244 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 11:24:29.586201 28259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:29.587757 28244 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 11:24:29.587970 28244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "b371b50bbc0045d382934fdde294d11d"
format_stamp: "Formatted at 2026-08-07 11:24:29 on dist-test-slave-2w3w"
server_key: "8e746bda55ebf70951334037ec2dbbd6"
server_key_iv: "ebcd906bb1dfb59e975425cfa265fdcb"
server_key_version: "encryptionkey@0"
I20260807 11:24:29.588205 28244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:24:29.606117 28244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:24:29.610487 28244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:24:29.610800 28244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:24:29.611994 28244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:24:29.613407 28244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:24:29.613516 28244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:29.613667 28244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:24:29.613739 28244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:29.648797 28244 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:45421
I20260807 11:24:29.648847 28375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:45421 every 8 connection(s)
I20260807 11:24:29.650146 28244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:24:29.657462 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 28244
I20260807 11:24:29.657725 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance
I20260807 11:24:29.658614 17172 external_mini_cluster.cc:1428] Setting key a45e41f07fc1dd237b196a1dc60791fc
Aug 07 11:24:29 dist-test-slave-2w3w krb5kdc[28133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101869, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:24:29.681181 28378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:29.652762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:59119 (local address 127.16.197.62:41365)
0807 11:24:29.653331 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:29.653339 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:29.654454 (+ 1115us) server_negotiation.cc:407] Connection header received
0807 11:24:29.657492 (+ 3038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:29.657500 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:29.657606 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:29.657717 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:29.660314 (+ 2597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:29.661488 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:29.664114 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:29.664909 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:29.665083 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:29.670789 (+ 5706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:29.670798 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:29.670803 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:24:29.670847 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:29.676561 (+ 5714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:29.677769 (+ 1208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:29.677778 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:29.677784 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:29.677970 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:29.678557 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:29.678562 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:29.678566 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:29.678874 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:29.679158 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:29.680364 (+ 1206us) server_negotiation.cc:299] Negotiation successful
0807 11:24:29.680667 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":312,"threads_started":1}
I20260807 11:24:29.682767 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:29.653255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:41365 (local address 127.16.197.1:59119)
0807 11:24:29.654243 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:29.654302 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:29.656975 (+ 2673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:29.658249 (+ 1274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:29.658265 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:29.659169 (+ 904us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:29.660089 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:29.660119 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:29.661681 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:29.661688 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:29.663949 (+ 2261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:29.663966 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:29.665461 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:29.665471 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:29.665854 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:29.666807 (+ 953us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:29.666845 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:29.670423 (+ 3578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:24:29.676814 (+ 6391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:24:29.676828 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:24:29.676853 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:24:29.677573 (+ 720us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:24:29.678145 (+ 572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:24:29.678153 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:24:29.678159 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:24:29.678438 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:24:29.679262 (+ 824us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:24:29.679274 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:24:29.679835 (+ 561us) client_negotiation.cc:769] Sending connection context
0807 11:24:29.680192 (+ 357us) client_negotiation.cc:240] Negotiation successful
0807 11:24:29.680816 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":374,"threads_started":1}
I20260807 11:24:29.685413 28376 heartbeater.cc:347] Connected to a master server at 127.16.197.62:41365
I20260807 11:24:29.685827 28376 heartbeater.cc:464] Registering TS with master...
I20260807 11:24:29.687011 28376 heartbeater.cc:511] Master 127.16.197.62:41365 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.16.197.2@KRBTEST.COM; defaulting to no policy
I20260807 11:24:29.691035 28181 ts_manager.cc:194] Registered new tserver with Master: b371b50bbc0045d382934fdde294d11d (127.16.197.1:45421)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.2@KRBTEST.COM" created.
I20260807 11:24:29.694020 28181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.1@KRBTEST.COM'} at 127.16.197.1:59119
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
I20260807 11:24:29.727077 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:0
--local_ip_for_outbound_sockets=127.16.197.2
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:41365
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:34589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:24:30.077900 28383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:24:30.078222 28383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:24:30.078321 28383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:24:30.086165 28383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:24:30.086295 28383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:24:30.086416 28383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:24:30.097215 28383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:34589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:41365
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:24:30.098632 28383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:24:30.100064 28383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:24:30.114702 28389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:30.114761 28392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:30.114791 28390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:24:30.115792 28383 server_base.cc:1034] running on GCE node
I20260807 11:24:30.116518 28383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:24:30.117630 28383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:24:30.118883 28383 hybrid_clock.cc:648] HybridClock initialized: now 1786101870118828 us; error 41 us; skew 500 ppm
I20260807 11:24:30.119345 28383 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:24:30 dist-test-slave-2w3w krb5kdc[28133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101870, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:24:30.123066 28383 init.cc:377] Logged in from keytab as kudu/127.16.197.2@KRBTEST.COM (short username kudu)
I20260807 11:24:30.124961 28383 webserver.cc:460] Webserver started at http://127.16.197.2:41081/ using document root <none> and password file <none>
I20260807 11:24:30.125850 28383 fs_manager.cc:359] Metadata directory not provided
I20260807 11:24:30.126017 28383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:24:30.126384 28383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:24:30.130098 28383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/data/instance:
uuid: "8fda3a64433f41e19645ab7d6e41dc4e"
format_stamp: "Formatted at 2026-08-07 11:24:30 on dist-test-slave-2w3w"
server_key: "e83adaa1264763bd3776a5a27e486073"
server_key_iv: "9d4a78f1e72d4594c36a118271c59e19"
server_key_version: "encryptionkey@0"
I20260807 11:24:30.131114 28383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance:
uuid: "8fda3a64433f41e19645ab7d6e41dc4e"
format_stamp: "Formatted at 2026-08-07 11:24:30 on dist-test-slave-2w3w"
server_key: "e83adaa1264763bd3776a5a27e486073"
server_key_iv: "9d4a78f1e72d4594c36a118271c59e19"
server_key_version: "encryptionkey@0"
I20260807 11:24:30.137043 28383 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 11:24:30.141232 28400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:30.142824 28383 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260807 11:24:30.143100 28383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "8fda3a64433f41e19645ab7d6e41dc4e"
format_stamp: "Formatted at 2026-08-07 11:24:30 on dist-test-slave-2w3w"
server_key: "e83adaa1264763bd3776a5a27e486073"
server_key_iv: "9d4a78f1e72d4594c36a118271c59e19"
server_key_version: "encryptionkey@0"
I20260807 11:24:30.143347 28383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:24:30.160835 28383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:24:30.165632 28383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:24:30.165972 28383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:24:30.167114 28383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:24:30.168596 28383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:24:30.168715 28383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:30.168886 28383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:24:30.168962 28383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:30.205619 28383 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:46825
I20260807 11:24:30.205670 28517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:46825 every 8 connection(s)
I20260807 11:24:30.207127 28383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:24:30.216722 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 28383
I20260807 11:24:30.216969 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance
I20260807 11:24:30.217514 17172 external_mini_cluster.cc:1428] Setting key c210f08b0c6d49971d5c8f8854624a59
Aug 07 11:24:30 dist-test-slave-2w3w krb5kdc[28133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101870, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:24:30.230845 28521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:30.209968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:60183 (local address 127.16.197.62:41365)
0807 11:24:30.210474 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:30.210478 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:30.211632 (+ 1154us) server_negotiation.cc:407] Connection header received
0807 11:24:30.213578 (+ 1946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:30.213584 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:30.213684 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:30.213813 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:30.215602 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:30.216373 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:30.218262 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:30.218736 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:30.218889 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:30.223353 (+ 4464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:30.223359 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:30.223362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:24:30.223439 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:30.226574 (+ 3135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:30.227725 (+ 1151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:30.227734 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:30.227741 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:30.227939 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:30.228525 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:30.228534 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:30.228540 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:30.229005 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:30.229169 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:30.230077 (+ 908us) server_negotiation.cc:299] Negotiation successful
0807 11:24:30.230385 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":250,"threads_started":1}
I20260807 11:24:30.231805 28522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:30.210586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:41365 (local address 127.16.197.2:60183)
0807 11:24:30.211448 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:30.211491 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:30.213264 (+ 1773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:30.214116 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:30.214125 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:30.214679 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:30.215407 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:30.215426 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:30.216564 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:30.216569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:30.218088 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:30.218102 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:30.219044 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:30.219053 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:30.219309 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:30.220039 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:30.220072 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:30.223045 (+ 2973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:24:30.226787 (+ 3742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:24:30.226801 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:24:30.226825 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:24:30.227545 (+ 720us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:24:30.228078 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:24:30.228085 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:24:30.228091 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:24:30.228382 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:24:30.229146 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:24:30.229157 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:24:30.229658 (+ 501us) client_negotiation.cc:769] Sending connection context
0807 11:24:30.229904 (+ 246us) client_negotiation.cc:240] Negotiation successful
0807 11:24:30.230355 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":396,"threads_started":1}
I20260807 11:24:30.233476 28519 heartbeater.cc:347] Connected to a master server at 127.16.197.62:41365
I20260807 11:24:30.233839 28519 heartbeater.cc:464] Registering TS with master...
I20260807 11:24:30.234686 28519 heartbeater.cc:511] Master 127.16.197.62:41365 requested a full tablet report, sending...
I20260807 11:24:30.236446 28181 ts_manager.cc:194] Registered new tserver with Master: 8fda3a64433f41e19645ab7d6e41dc4e (127.16.197.2:46825)
I20260807 11:24:30.237393 28181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.2@KRBTEST.COM'} at 127.16.197.2:60183
WARNING: no policy specified for kudu/127.16.197.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
I20260807 11:24:30.282021 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:0
--local_ip_for_outbound_sockets=127.16.197.3
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:41365
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:34589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:24:30.638073 28525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:24:30.638479 28525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:24:30.638686 28525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:24:30.646754 28525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:24:30.646900 28525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:24:30.647125 28525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:24:30.658627 28525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:34589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:41365
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:24:30.660125 28525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:24:30.661589 28525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:24:30.676867 28531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:30.677006 28534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:24:30.676949 28532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:24:30.678223 28525 server_base.cc:1034] running on GCE node
I20260807 11:24:30.679332 28525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:24:30.680683 28525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:24:30.682029 28525 hybrid_clock.cc:648] HybridClock initialized: now 1786101870681952 us; error 76 us; skew 500 ppm
I20260807 11:24:30.682458 28525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:24:30 dist-test-slave-2w3w krb5kdc[28133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101870, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:24:30.686393 28525 init.cc:377] Logged in from keytab as kudu/127.16.197.3@KRBTEST.COM (short username kudu)
I20260807 11:24:30.688591 28525 webserver.cc:460] Webserver started at http://127.16.197.3:43067/ using document root <none> and password file <none>
I20260807 11:24:30.689575 28525 fs_manager.cc:359] Metadata directory not provided
I20260807 11:24:30.689765 28525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:24:30.690191 28525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:24:30.694093 28525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/data/instance:
uuid: "3aef4714265749609a134a48c39026ad"
format_stamp: "Formatted at 2026-08-07 11:24:30 on dist-test-slave-2w3w"
server_key: "59ba8efc59e5b2060a3f3a98bf90d2b5"
server_key_iv: "c16ebdbcc554bc6c8a742997fdbd8a0e"
server_key_version: "encryptionkey@0"
I20260807 11:24:30.695152 28525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance:
uuid: "3aef4714265749609a134a48c39026ad"
format_stamp: "Formatted at 2026-08-07 11:24:30 on dist-test-slave-2w3w"
server_key: "59ba8efc59e5b2060a3f3a98bf90d2b5"
server_key_iv: "c16ebdbcc554bc6c8a742997fdbd8a0e"
server_key_version: "encryptionkey@0"
I20260807 11:24:30.698891 28376 heartbeater.cc:503] Master 127.16.197.62:41365 was elected leader, sending a full tablet report...
I20260807 11:24:30.702015 28525 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 11:24:30.706799 28541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:30.708580 28525 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 11:24:30.708822 28525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "3aef4714265749609a134a48c39026ad"
format_stamp: "Formatted at 2026-08-07 11:24:30 on dist-test-slave-2w3w"
server_key: "59ba8efc59e5b2060a3f3a98bf90d2b5"
server_key_iv: "c16ebdbcc554bc6c8a742997fdbd8a0e"
server_key_version: "encryptionkey@0"
I20260807 11:24:30.709076 28525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:24:30.745500 28525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:24:30.750387 28525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:24:30.750716 28525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:24:30.751930 28525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:24:30.753422 28525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:24:30.753535 28525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:30.753700 28525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:24:30.753775 28525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:24:30.790465 28525 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:40715
I20260807 11:24:30.790562 28660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:40715 every 8 connection(s)
I20260807 11:24:30.792039 28525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:24:30.792328 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 28525
I20260807 11:24:30.792554 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance
I20260807 11:24:30.793262 17172 external_mini_cluster.cc:1428] Setting key 7390a4d673cf982c201510b295baf89f
Aug 07 11:24:30 dist-test-slave-2w3w krb5kdc[28133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101870, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:24:30.815356 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:30.795491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:42909 (local address 127.16.197.62:41365)
0807 11:24:30.795989 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:30.795994 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:30.797299 (+ 1305us) server_negotiation.cc:407] Connection header received
0807 11:24:30.799371 (+ 2072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:30.799376 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:30.799456 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:30.799568 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:30.801225 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:30.802065 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:30.803746 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:30.804157 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:30.804262 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:30.808258 (+ 3996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:30.808264 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:30.808268 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:24:30.808307 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:30.811402 (+ 3095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:30.812273 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:30.812277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:30.812280 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:30.812412 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:30.812919 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:30.812924 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:30.812926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:30.813236 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:30.813443 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:30.814297 (+ 854us) server_negotiation.cc:299] Negotiation successful
0807 11:24:30.814643 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":236,"threads_started":1}
I20260807 11:24:30.816565 28664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:30.796224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:41365 (local address 127.16.197.3:42909)
0807 11:24:30.797116 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:30.797175 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:30.799054 (+ 1879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:30.799803 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:30.799812 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:30.800392 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:24:30.801018 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:30.801038 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:30.802206 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:30.802210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:30.803598 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:30.803612 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:30.804365 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:30.804370 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:30.804578 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:30.805308 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:24:30.805336 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:24:30.807984 (+ 2648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:24:30.811593 (+ 3609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:24:30.811602 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:24:30.811619 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:24:30.812141 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:24:30.812514 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:24:30.812517 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:24:30.812519 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:24:30.812727 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:24:30.813412 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:24:30.813419 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:24:30.813897 (+ 478us) client_negotiation.cc:769] Sending connection context
0807 11:24:30.814209 (+ 312us) client_negotiation.cc:240] Negotiation successful
0807 11:24:30.814941 (+ 732us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":379,"threads_started":1}
I20260807 11:24:30.818080 28661 heartbeater.cc:347] Connected to a master server at 127.16.197.62:41365
I20260807 11:24:30.818426 28661 heartbeater.cc:464] Registering TS with master...
I20260807 11:24:30.819308 28661 heartbeater.cc:511] Master 127.16.197.62:41365 requested a full tablet report, sending...
I20260807 11:24:30.821363 28181 ts_manager.cc:194] Registered new tserver with Master: 3aef4714265749609a134a48c39026ad (127.16.197.3:40715)
I20260807 11:24:30.822619 28181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.3@KRBTEST.COM'} at 127.16.197.3:42909
I20260807 11:24:30.822731 17172 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 11:24:30.842087 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:30.827911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48058 (local address 127.16.197.62:41365)
0807 11:24:30.828119 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:30.828127 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:30.828709 (+ 582us) server_negotiation.cc:407] Connection header received
0807 11:24:30.828956 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:30.828967 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:30.829050 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:30.829161 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:30.830689 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:30.831535 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:30.832832 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:30.833273 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:30.833430 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:30.836093 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:30.836098 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:30.836102 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:24:30.836141 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:30.839093 (+ 2952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:30.839894 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:30.839900 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:30.839904 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:30.840056 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:30.840471 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:30.840477 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:30.840481 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:30.840831 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:30.841114 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:30.841498 (+ 384us) server_negotiation.cc:299] Negotiation successful
0807 11:24:30.841739 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 11:24:30.846721 28181 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48058:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 11:24:30.854274 28181 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:24:30.892546 28677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:30.869798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50222 (local address 127.16.197.1:45421)
0807 11:24:30.879846 (+ 10048us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:30.879899 (+ 53us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:30.882349 (+ 2450us) server_negotiation.cc:407] Connection header received
0807 11:24:30.882507 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:30.882521 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:30.882938 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:30.883255 (+ 317us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:24:30.884274 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:30.886254 (+ 1980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:30.889435 (+ 3181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:30.891116 (+ 1681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:30.891294 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:30.891639 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:30.891984 (+ 345us) server_negotiation.cc:299] Negotiation successful
0807 11:24:30.892169 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9800,"thread_start_us":9601,"threads_started":1}
I20260807 11:24:30.892606 28678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:30.870249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:46825 (local address 127.0.0.1:44450)
0807 11:24:30.882507 (+ 12258us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:30.882533 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:30.882678 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:30.883783 (+ 1105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:30.883789 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:30.883820 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:24:30.884335 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:30.884345 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:30.886723 (+ 2378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:30.886729 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:30.888678 (+ 1949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:30.888724 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:30.891060 (+ 2336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:30.891069 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:30.891544 (+ 475us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:30.891639 (+ 95us) client_negotiation.cc:769] Sending connection context
0807 11:24:30.891842 (+ 203us) client_negotiation.cc:240] Negotiation successful
0807 11:24:30.892128 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12121,"mutex_wait_us":61,"thread_start_us":11937,"threads_started":1}
I20260807 11:24:30.892761 28679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:30.870547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44450 (local address 127.16.197.2:46825)
0807 11:24:30.881704 (+ 11157us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:30.881718 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:30.882835 (+ 1117us) server_negotiation.cc:407] Connection header received
0807 11:24:30.883024 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:30.883031 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:30.883287 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:30.883524 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:24:30.884649 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:30.886509 (+ 1860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:30.888915 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:30.890452 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:30.890610 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:30.890836 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:30.892112 (+ 1276us) server_negotiation.cc:299] Negotiation successful
0807 11:24:30.892332 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10888,"thread_start_us":11185,"threads_started":1}
I20260807 11:24:30.893633 28674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:30.868593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:40715 (local address 127.0.0.1:48600)
0807 11:24:30.869782 (+ 1189us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:30.869824 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:30.870018 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:30.884867 (+ 14849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:30.884875 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:30.884910 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:24:30.885407 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:30.885426 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:30.887467 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:30.887478 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:30.889805 (+ 2327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:30.889821 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:30.892365 (+ 2544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:30.892372 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:30.892567 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:30.892604 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 11:24:30.893118 (+ 514us) client_negotiation.cc:240] Negotiation successful
0807 11:24:30.893268 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":937,"thread_start_us":786,"threads_started":1}
I20260807 11:24:30.894043 28675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:30.868884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48600 (local address 127.16.197.3:40715)
0807 11:24:30.884183 (+ 15299us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:30.884192 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:30.884231 (+ 39us) server_negotiation.cc:407] Connection header received
0807 11:24:30.884339 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:30.884350 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:30.884627 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:30.884855 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:24:30.885640 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:30.887285 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:30.890024 (+ 2739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:30.892084 (+ 2060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:30.893117 (+ 1033us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:30.893362 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:30.893454 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 11:24:30.893636 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15104,"thread_start_us":14915,"threads_started":1}
I20260807 11:24:30.894570 28676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:30.868985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:45421 (local address 127.0.0.1:50222)
0807 11:24:30.881968 (+ 12983us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:30.881992 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:30.882176 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:30.883454 (+ 1278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:30.883464 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:30.883499 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:24:30.884011 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:30.884042 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:30.886463 (+ 2421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:30.886468 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:30.889201 (+ 2733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:30.889218 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:30.891428 (+ 2210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:30.891435 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:30.891758 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:30.891795 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 11:24:30.891924 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 11:24:30.894190 (+ 2266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12828,"mutex_wait_us":2192,"thread_start_us":12657,"threads_started":1}
I20260807 11:24:30.897536 28451 tablet_service.cc:1467] Processing CreateTablet for tablet 40f04ef0e090444e8ad087d3735c5680 (DEFAULT_TABLE table=test-table [id=2d5f8100547e46d1870df236e24c0fcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:24:30.897955 28594 tablet_service.cc:1467] Processing CreateTablet for tablet 40f04ef0e090444e8ad087d3735c5680 (DEFAULT_TABLE table=test-table [id=2d5f8100547e46d1870df236e24c0fcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:24:30.898351 28308 tablet_service.cc:1467] Processing CreateTablet for tablet 40f04ef0e090444e8ad087d3735c5680 (DEFAULT_TABLE table=test-table [id=2d5f8100547e46d1870df236e24c0fcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:24:30.899308 28451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40f04ef0e090444e8ad087d3735c5680. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:24:30.899358 28594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40f04ef0e090444e8ad087d3735c5680. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:24:30.899788 28308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40f04ef0e090444e8ad087d3735c5680. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:24:30.913159 28680 tablet_bootstrap.cc:492] T 40f04ef0e090444e8ad087d3735c5680 P 3aef4714265749609a134a48c39026ad: Bootstrap starting.
I20260807 11:24:30.917958 28680 tablet_bootstrap.cc:654] T 40f04ef0e090444e8ad087d3735c5680 P 3aef4714265749609a134a48c39026ad: Neither blocks nor log segments found. Creating new log.
I20260807 11:24:30.919276 28680 log.cc:824] T 40f04ef0e090444e8ad087d3735c5680 P 3aef4714265749609a134a48c39026ad: Log is configured to *not* fsync() on all Append() calls
I20260807 11:24:30.922209 28682 tablet_bootstrap.cc:492] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d: Bootstrap starting.
I20260807 11:24:30.922780 28680 tablet_bootstrap.cc:492] T 40f04ef0e090444e8ad087d3735c5680 P 3aef4714265749609a134a48c39026ad: No bootstrap required, opened a new log
I20260807 11:24:30.923096 28681 tablet_bootstrap.cc:492] T 40f04ef0e090444e8ad087d3735c5680 P 8fda3a64433f41e19645ab7d6e41dc4e: Bootstrap starting.
I20260807 11:24:30.923170 28680 ts_tablet_manager.cc:1397] T 40f04ef0e090444e8ad087d3735c5680 P 3aef4714265749609a134a48c39026ad: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260807 11:24:30.925860 28682 tablet_bootstrap.cc:654] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d: Neither blocks nor log segments found. Creating new log.
I20260807 11:24:30.927142 28682 log.cc:824] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d: Log is configured to *not* fsync() on all Append() calls
I20260807 11:24:30.927552 28681 tablet_bootstrap.cc:654] T 40f04ef0e090444e8ad087d3735c5680 P 8fda3a64433f41e19645ab7d6e41dc4e: Neither blocks nor log segments found. Creating new log.
I20260807 11:24:30.928972 28681 log.cc:824] T 40f04ef0e090444e8ad087d3735c5680 P 8fda3a64433f41e19645ab7d6e41dc4e: Log is configured to *not* fsync() on all Append() calls
I20260807 11:24:30.931463 28682 tablet_bootstrap.cc:492] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d: No bootstrap required, opened a new log
I20260807 11:24:30.931771 28682 ts_tablet_manager.cc:1397] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 11:24:30.932251 28681 tablet_bootstrap.cc:492] T 40f04ef0e090444e8ad087d3735c5680 P 8fda3a64433f41e19645ab7d6e41dc4e: No bootstrap required, opened a new log
I20260807 11:24:30.932540 28681 ts_tablet_manager.cc:1397] T 40f04ef0e090444e8ad087d3735c5680 P 8fda3a64433f41e19645ab7d6e41dc4e: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260807 11:24:30.935173 28680 raft_consensus.cc:348] T 40f04ef0e090444e8ad087d3735c5680 P 3aef4714265749609a134a48c39026ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aef4714265749609a134a48c39026ad" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40715 } } peers { permanent_uuid: "b371b50bbc0045d382934fdde294d11d" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45421 } } peers { permanent_uuid: "8fda3a64433f41e19645ab7d6e41dc4e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46825 } }
I20260807 11:24:30.935675 28680 raft_consensus.cc:374] T 40f04ef0e090444e8ad087d3735c5680 P 3aef4714265749609a134a48c39026ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:24:30.935774 28680 raft_consensus.cc:729] T 40f04ef0e090444e8ad087d3735c5680 P 3aef4714265749609a134a48c39026ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aef4714265749609a134a48c39026ad, State: Initialized, Role: FOLLOWER
I20260807 11:24:30.936545 28680 consensus_queue.cc:260] T 40f04ef0e090444e8ad087d3735c5680 P 3aef4714265749609a134a48c39026ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aef4714265749609a134a48c39026ad" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40715 } } peers { permanent_uuid: "b371b50bbc0045d382934fdde294d11d" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45421 } } peers { permanent_uuid: "8fda3a64433f41e19645ab7d6e41dc4e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46825 } }
I20260807 11:24:30.938380 28661 heartbeater.cc:503] Master 127.16.197.62:41365 was elected leader, sending a full tablet report...
I20260807 11:24:30.939111 28680 ts_tablet_manager.cc:1428] T 40f04ef0e090444e8ad087d3735c5680 P 3aef4714265749609a134a48c39026ad: Time spent starting tablet: real 0.016s user 0.011s sys 0.004s
I20260807 11:24:30.941169 28682 raft_consensus.cc:348] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aef4714265749609a134a48c39026ad" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40715 } } peers { permanent_uuid: "b371b50bbc0045d382934fdde294d11d" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45421 } } peers { permanent_uuid: "8fda3a64433f41e19645ab7d6e41dc4e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46825 } }
I20260807 11:24:30.941587 28682 raft_consensus.cc:374] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:24:30.941731 28682 raft_consensus.cc:729] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b371b50bbc0045d382934fdde294d11d, State: Initialized, Role: FOLLOWER
I20260807 11:24:30.942386 28682 consensus_queue.cc:260] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aef4714265749609a134a48c39026ad" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40715 } } peers { permanent_uuid: "b371b50bbc0045d382934fdde294d11d" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45421 } } peers { permanent_uuid: "8fda3a64433f41e19645ab7d6e41dc4e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46825 } }
I20260807 11:24:30.943415 28681 raft_consensus.cc:348] T 40f04ef0e090444e8ad087d3735c5680 P 8fda3a64433f41e19645ab7d6e41dc4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aef4714265749609a134a48c39026ad" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40715 } } peers { permanent_uuid: "b371b50bbc0045d382934fdde294d11d" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45421 } } peers { permanent_uuid: "8fda3a64433f41e19645ab7d6e41dc4e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46825 } }
I20260807 11:24:30.943902 28681 raft_consensus.cc:374] T 40f04ef0e090444e8ad087d3735c5680 P 8fda3a64433f41e19645ab7d6e41dc4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:24:30.944023 28681 raft_consensus.cc:729] T 40f04ef0e090444e8ad087d3735c5680 P 8fda3a64433f41e19645ab7d6e41dc4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fda3a64433f41e19645ab7d6e41dc4e, State: Initialized, Role: FOLLOWER
I20260807 11:24:30.944392 28682 ts_tablet_manager.cc:1428] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d: Time spent starting tablet: real 0.012s user 0.009s sys 0.003s
I20260807 11:24:30.944741 28681 consensus_queue.cc:260] T 40f04ef0e090444e8ad087d3735c5680 P 8fda3a64433f41e19645ab7d6e41dc4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aef4714265749609a134a48c39026ad" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40715 } } peers { permanent_uuid: "b371b50bbc0045d382934fdde294d11d" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45421 } } peers { permanent_uuid: "8fda3a64433f41e19645ab7d6e41dc4e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46825 } }
I20260807 11:24:30.946197 28519 heartbeater.cc:503] Master 127.16.197.62:41365 was elected leader, sending a full tablet report...
I20260807 11:24:30.946830 28681 ts_tablet_manager.cc:1428] T 40f04ef0e090444e8ad087d3735c5680 P 8fda3a64433f41e19645ab7d6e41dc4e: Time spent starting tablet: real 0.014s user 0.005s sys 0.009s
W20260807 11:24:30.959311 28520 tablet.cc:2369] T 40f04ef0e090444e8ad087d3735c5680 P 8fda3a64433f41e19645ab7d6e41dc4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:24:30.993749 28688 raft_consensus.cc:482] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:24:30.994426 28688 raft_consensus.cc:504] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aef4714265749609a134a48c39026ad" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40715 } } peers { permanent_uuid: "b371b50bbc0045d382934fdde294d11d" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45421 } } peers { permanent_uuid: "8fda3a64433f41e19645ab7d6e41dc4e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46825 } }
I20260807 11:24:30.996589 28688 leader_election.cc:288] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3aef4714265749609a134a48c39026ad (127.16.197.3:40715), 8fda3a64433f41e19645ab7d6e41dc4e (127.16.197.2:46825)
I20260807 11:24:31.005611 28675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:30.997140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:36917 (local address 127.16.197.3:40715)
0807 11:24:30.997620 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:30.997629 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:30.998258 (+ 629us) server_negotiation.cc:407] Connection header received
0807 11:24:30.998568 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:30.998574 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:30.998663 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:30.998745 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:24:30.999663 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:31.000730 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:31.003427 (+ 2697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:31.004577 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:31.004746 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:31.004806 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:31.005249 (+ 443us) server_negotiation.cc:299] Negotiation successful
0807 11:24:31.005374 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210}
I20260807 11:24:31.005959 28690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:30.996872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:40715 (local address 127.16.197.1:36917)
0807 11:24:30.998067 (+ 1195us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:30.998087 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:30.998310 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:30.999021 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:30.999027 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:30.999068 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:24:30.999462 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:30.999480 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:31.000995 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:31.001003 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:31.003221 (+ 2218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:31.003236 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:31.004845 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:31.004853 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:31.005048 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:31.005095 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 11:24:31.005209 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 11:24:31.005366 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1054,"mutex_wait_us":31,"thread_start_us":594,"threads_started":1}
I20260807 11:24:31.006582 28679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:30.998424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:36747 (local address 127.16.197.2:46825)
0807 11:24:30.998663 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:30.998670 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:30.999008 (+ 338us) server_negotiation.cc:407] Connection header received
0807 11:24:30.999317 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:30.999325 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:30.999432 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:30.999549 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:24:31.000391 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:31.001981 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:31.004310 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:31.005485 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:31.005660 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:31.005734 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:31.006232 (+ 498us) server_negotiation.cc:299] Negotiation successful
0807 11:24:31.006393 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260807 11:24:31.006742 28691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:30.998112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:46825 (local address 127.16.197.1:36747)
0807 11:24:30.998879 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0807 11:24:30.998895 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 11:24:30.999143 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:24:30.999642 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:24:30.999647 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:24:30.999678 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:24:31.000169 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:31.000183 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:31.002174 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:31.002179 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:31.004105 (+ 1926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:24:31.004121 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:24:31.005742 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:24:31.005751 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:24:31.006004 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:31.006030 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 11:24:31.006149 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 11:24:31.006301 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"mutex_wait_us":5,"spinlock_wait_cycles":22400,"thread_start_us":397,"threads_started":1}
I20260807 11:24:31.007287 28614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40f04ef0e090444e8ad087d3735c5680" candidate_uuid: "b371b50bbc0045d382934fdde294d11d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aef4714265749609a134a48c39026ad" is_pre_election: true
I20260807 11:24:31.007966 28614 raft_consensus.cc:2454] T 40f04ef0e090444e8ad087d3735c5680 P 3aef4714265749609a134a48c39026ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b371b50bbc0045d382934fdde294d11d in term 0.
I20260807 11:24:31.007941 28471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40f04ef0e090444e8ad087d3735c5680" candidate_uuid: "b371b50bbc0045d382934fdde294d11d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fda3a64433f41e19645ab7d6e41dc4e" is_pre_election: true
I20260807 11:24:31.008499 28471 raft_consensus.cc:2454] T 40f04ef0e090444e8ad087d3735c5680 P 8fda3a64433f41e19645ab7d6e41dc4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b371b50bbc0045d382934fdde294d11d in term 0.
I20260807 11:24:31.008803 28260 leader_election.cc:302] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aef4714265749609a134a48c39026ad, b371b50bbc0045d382934fdde294d11d; no voters:
I20260807 11:24:31.009307 28688 raft_consensus.cc:2781] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:24:31.009477 28688 raft_consensus.cc:482] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:24:31.009601 28688 raft_consensus.cc:3037] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:24:31.012245 28688 raft_consensus.cc:504] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aef4714265749609a134a48c39026ad" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40715 } } peers { permanent_uuid: "b371b50bbc0045d382934fdde294d11d" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45421 } } peers { permanent_uuid: "8fda3a64433f41e19645ab7d6e41dc4e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46825 } }
I20260807 11:24:31.012934 28688 leader_election.cc:288] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d [CANDIDATE]: Term 1 election: Requested vote from peers 3aef4714265749609a134a48c39026ad (127.16.197.3:40715), 8fda3a64433f41e19645ab7d6e41dc4e (127.16.197.2:46825)
I20260807 11:24:31.013605 28614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40f04ef0e090444e8ad087d3735c5680" candidate_uuid: "b371b50bbc0045d382934fdde294d11d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aef4714265749609a134a48c39026ad"
I20260807 11:24:31.013757 28471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40f04ef0e090444e8ad087d3735c5680" candidate_uuid: "b371b50bbc0045d382934fdde294d11d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fda3a64433f41e19645ab7d6e41dc4e"
I20260807 11:24:31.013854 28614 raft_consensus.cc:3037] T 40f04ef0e090444e8ad087d3735c5680 P 3aef4714265749609a134a48c39026ad [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:24:31.014050 28471 raft_consensus.cc:3037] T 40f04ef0e090444e8ad087d3735c5680 P 8fda3a64433f41e19645ab7d6e41dc4e [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:24:31.016402 28614 raft_consensus.cc:2454] T 40f04ef0e090444e8ad087d3735c5680 P 3aef4714265749609a134a48c39026ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b371b50bbc0045d382934fdde294d11d in term 1.
I20260807 11:24:31.016613 28471 raft_consensus.cc:2454] T 40f04ef0e090444e8ad087d3735c5680 P 8fda3a64433f41e19645ab7d6e41dc4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b371b50bbc0045d382934fdde294d11d in term 1.
I20260807 11:24:31.016954 28260 leader_election.cc:302] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aef4714265749609a134a48c39026ad, b371b50bbc0045d382934fdde294d11d; no voters:
I20260807 11:24:31.017318 28688 raft_consensus.cc:2781] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:24:31.017812 28688 raft_consensus.cc:686] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d [term 1 LEADER]: Becoming Leader. State: Replica: b371b50bbc0045d382934fdde294d11d, State: Running, Role: LEADER
I20260807 11:24:31.018369 28688 consensus_queue.cc:237] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aef4714265749609a134a48c39026ad" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40715 } } peers { permanent_uuid: "b371b50bbc0045d382934fdde294d11d" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45421 } } peers { permanent_uuid: "8fda3a64433f41e19645ab7d6e41dc4e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46825 } }
I20260807 11:24:31.023190 28181 catalog_manager.cc:5495] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d reported cstate change: term changed from 0 to 1, leader changed from <none> to b371b50bbc0045d382934fdde294d11d (127.16.197.1). New cstate: current_term: 1 leader_uuid: "b371b50bbc0045d382934fdde294d11d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aef4714265749609a134a48c39026ad" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b371b50bbc0045d382934fdde294d11d" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8fda3a64433f41e19645ab7d6e41dc4e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46825 } health_report { overall_health: UNKNOWN } } }
W20260807 11:24:31.044230 28662 tablet.cc:2369] T 40f04ef0e090444e8ad087d3735c5680 P 3aef4714265749609a134a48c39026ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:24:31.057210 28677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:31.050357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50234 (local address 127.16.197.1:45421)
0807 11:24:31.051409 (+ 1052us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:31.051414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:31.051431 (+ 17us) server_negotiation.cc:407] Connection header received
0807 11:24:31.051539 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:31.051543 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:31.051645 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:31.051861 (+ 216us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:24:31.052347 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:31.053099 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:31.054579 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:31.055126 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:31.055274 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:31.055905 (+ 631us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:24:31.056489 (+ 584us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:24:31.056661 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:31.056806 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 11:24:31.056905 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":875}
I20260807 11:24:31.067811 28614 raft_consensus.cc:1261] T 40f04ef0e090444e8ad087d3735c5680 P 3aef4714265749609a134a48c39026ad [term 1 FOLLOWER]: Refusing update from remote peer b371b50bbc0045d382934fdde294d11d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:24:31.067842 28471 raft_consensus.cc:1261] T 40f04ef0e090444e8ad087d3735c5680 P 8fda3a64433f41e19645ab7d6e41dc4e [term 1 FOLLOWER]: Refusing update from remote peer b371b50bbc0045d382934fdde294d11d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:24:31.069003 28688 consensus_queue.cc:1035] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fda3a64433f41e19645ab7d6e41dc4e" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:24:31.069360 28692 consensus_queue.cc:1035] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aef4714265749609a134a48c39026ad" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:24:31.084105 28695 mvcc.cc:205] Tried to move back new op lower bound from 7315873263879405568 to 7315873263695421440. Current Snapshot: MvccSnapshot[applied={T|T < 7315873263879405568}]
I20260807 11:24:31.101336 28181 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:24:31.101620 28181 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:24:31.105723 28181 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e2844a6c29344025abf64e1c08512f53: Sending DeleteTablet for 3 replicas of tablet 40f04ef0e090444e8ad087d3735c5680
I20260807 11:24:31.107273 28451 tablet_service.cc:1514] Processing DeleteTablet for tablet 40f04ef0e090444e8ad087d3735c5680 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:24:31 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:44450
I20260807 11:24:31.107273 28594 tablet_service.cc:1514] Processing DeleteTablet for tablet 40f04ef0e090444e8ad087d3735c5680 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:24:31 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:48600
I20260807 11:24:31.107278 28308 tablet_service.cc:1514] Processing DeleteTablet for tablet 40f04ef0e090444e8ad087d3735c5680 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:24:31 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:50222
I20260807 11:24:31.109505 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 28244
I20260807 11:24:31.119966 28706 tablet_replica.cc:330] T 40f04ef0e090444e8ad087d3735c5680 P 3aef4714265749609a134a48c39026ad: stopping tablet replica
I20260807 11:24:31.120801 28705 tablet_replica.cc:330] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d: stopping tablet replica
I20260807 11:24:31.121740 28705 raft_consensus.cc:2229] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d [term 1 LEADER]: Raft consensus shutting down.
I20260807 11:24:31.122478 28706 raft_consensus.cc:2229] T 40f04ef0e090444e8ad087d3735c5680 P 3aef4714265749609a134a48c39026ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:24:31.123306 28706 raft_consensus.cc:2258] T 40f04ef0e090444e8ad087d3735c5680 P 3aef4714265749609a134a48c39026ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:24:31.124385 28705 raft_consensus.cc:2258] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:24:31.126672 28707 tablet_replica.cc:330] T 40f04ef0e090444e8ad087d3735c5680 P 8fda3a64433f41e19645ab7d6e41dc4e: stopping tablet replica
I20260807 11:24:31.127907 28706 ts_tablet_manager.cc:1905] T 40f04ef0e090444e8ad087d3735c5680 P 3aef4714265749609a134a48c39026ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:24:31.128587 28707 raft_consensus.cc:2229] T 40f04ef0e090444e8ad087d3735c5680 P 8fda3a64433f41e19645ab7d6e41dc4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:24:31.129180 28707 raft_consensus.cc:2258] T 40f04ef0e090444e8ad087d3735c5680 P 8fda3a64433f41e19645ab7d6e41dc4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:24:31.129741 28705 ts_tablet_manager.cc:1905] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 11:24:31 dist-test-slave-2w3w krb5kdc[28133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.1@KRBTEST.COM
I20260807 11:24:31.135874 28706 ts_tablet_manager.cc:1918] T 40f04ef0e090444e8ad087d3735c5680 P 3aef4714265749609a134a48c39026ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:24:31.136081 28706 log.cc:1196] T 40f04ef0e090444e8ad087d3735c5680 P 3aef4714265749609a134a48c39026ad: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-2/wal/wals/40f04ef0e090444e8ad087d3735c5680
I20260807 11:24:31.136215 28707 ts_tablet_manager.cc:1905] T 40f04ef0e090444e8ad087d3735c5680 P 8fda3a64433f41e19645ab7d6e41dc4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:24:31.136654 28706 ts_tablet_manager.cc:1939] T 40f04ef0e090444e8ad087d3735c5680 P 3aef4714265749609a134a48c39026ad: Deleting consensus metadata
I20260807 11:24:31.137809 28705 ts_tablet_manager.cc:1918] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:24:31.138007 28705 log.cc:1196] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-0/wal/wals/40f04ef0e090444e8ad087d3735c5680
I20260807 11:24:31.138978 28164 catalog_manager.cc:4841] TS 3aef4714265749609a134a48c39026ad (127.16.197.3:40715): tablet 40f04ef0e090444e8ad087d3735c5680 (table test-table [id=2d5f8100547e46d1870df236e24c0fcc]) successfully deleted
I20260807 11:24:31.138757 28705 ts_tablet_manager.cc:1939] T 40f04ef0e090444e8ad087d3735c5680 P b371b50bbc0045d382934fdde294d11d: Deleting consensus metadata
I20260807 11:24:31.142333 28166 catalog_manager.cc:4841] TS b371b50bbc0045d382934fdde294d11d (127.16.197.1:45421): tablet 40f04ef0e090444e8ad087d3735c5680 (table test-table [id=2d5f8100547e46d1870df236e24c0fcc]) successfully deleted
I20260807 11:24:31.145844 28707 ts_tablet_manager.cc:1918] T 40f04ef0e090444e8ad087d3735c5680 P 8fda3a64433f41e19645ab7d6e41dc4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:24:31.146145 28707 log.cc:1196] T 40f04ef0e090444e8ad087d3735c5680 P 8fda3a64433f41e19645ab7d6e41dc4e: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786101765888975-17172-0/minicluster-data/ts-1/wal/wals/40f04ef0e090444e8ad087d3735c5680
I20260807 11:24:31.146328 28677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:31.111105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50242 (local address 127.16.197.1:45421)
0807 11:24:31.111310 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:31.111317 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:31.116359 (+ 5042us) server_negotiation.cc:407] Connection header received
0807 11:24:31.116632 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:31.116639 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:31.116733 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:31.116858 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:31.122372 (+ 5514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:31.123717 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:31.126701 (+ 2984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:31.127545 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:31.127667 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:31.138561 (+ 10894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:31.138584 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:31.138606 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:24:31.138667 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:31.143181 (+ 4514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:31.144021 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:31.144028 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:31.144031 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:31.144169 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:31.144620 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:31.144629 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:31.144631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:31.145019 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:31.145334 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:31.145682 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 11:24:31.145935 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 11:24:31.147022 28707 ts_tablet_manager.cc:1939] T 40f04ef0e090444e8ad087d3735c5680 P 8fda3a64433f41e19645ab7d6e41dc4e: Deleting consensus metadata
I20260807 11:24:31.147246 28371 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:24:31.149397 28164 catalog_manager.cc:4841] TS 8fda3a64433f41e19645ab7d6e41dc4e (127.16.197.2:46825): tablet 40f04ef0e090444e8ad087d3735c5680 (table test-table [id=2d5f8100547e46d1870df236e24c0fcc]) successfully deleted
I20260807 11:24:31.252231 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 28244
I20260807 11:24:31.272110 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 28383
Aug 07 11:24:31 dist-test-slave-2w3w krb5kdc[28133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.2@KRBTEST.COM
I20260807 11:24:31.287955 28679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:31.272917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44462 (local address 127.16.197.2:46825)
0807 11:24:31.273120 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:31.273124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:31.273143 (+ 19us) server_negotiation.cc:407] Connection header received
0807 11:24:31.273264 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:31.273268 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:31.273332 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:31.273417 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:31.274640 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:31.275468 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:31.276676 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:31.277058 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:31.277154 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:31.281546 (+ 4392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:31.281564 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:31.281580 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:24:31.281628 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:31.284827 (+ 3199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:31.285591 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:31.285599 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:31.285601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:31.285741 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:31.286098 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:31.286101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:31.286103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:31.286528 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:31.286757 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:31.287412 (+ 655us) server_negotiation.cc:299] Negotiation successful
0807 11:24:31.287685 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 11:24:31.288709 28513 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:24:31.382584 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 28383
I20260807 11:24:31.402266 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 28525
Aug 07 11:24:31 dist-test-slave-2w3w krb5kdc[28133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.3@KRBTEST.COM
I20260807 11:24:31.418637 28675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:24:31.403221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48606 (local address 127.16.197.3:40715)
0807 11:24:31.403419 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 11:24:31.403424 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:24:31.403444 (+ 20us) server_negotiation.cc:407] Connection header received
0807 11:24:31.403598 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:24:31.403602 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:24:31.403669 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:24:31.403757 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:24:31.405121 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:31.405852 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:31.407354 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:24:31.407764 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:24:31.407866 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:24:31.412573 (+ 4707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:24:31.412591 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:24:31.412607 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:24:31.412653 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:24:31.415929 (+ 3276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:31.416702 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:31.416709 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:31.416712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:31.416856 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:24:31.417225 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:24:31.417228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:24:31.417230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:24:31.417606 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:24:31.417828 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0807 11:24:31.418180 (+ 352us) server_negotiation.cc:299] Negotiation successful
0807 11:24:31.418359 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 11:24:31.419515 28656 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:24:31.511081 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 28525
I20260807 11:24:31.530364 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 28147
I20260807 11:24:31.531563 28211 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:24:31.646119 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 28147
2026-08-07T11:24:31Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4564 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:24:31 dist-test-slave-2w3w krb5kdc[28717](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:24:31 dist-test-slave-2w3w krb5kdc[28717](info): set up 2 sockets
Aug 07 11:24:31 dist-test-slave-2w3w krb5kdc[28717](info): commencing operation
krb5kdc: starting...
W20260807 11:24:32.794198 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.004s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:24:32 dist-test-slave-2w3w krb5kdc[28717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:24:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:24:33Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.16.197.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.16.197.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.16.197.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/rangeradmin_127.16.197.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.16.197.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/rangeradmin_127.16.197.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.16.197.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.16.197.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.16.197.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/rangerlookup_127.16.197.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.16.197.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/rangerlookup_127.16.197.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.16.197.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.16.197.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.16.197.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/HTTP_127.16.197.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.16.197.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/HTTP_127.16.197.20@KRBTEST.COM.keytab.
I20260807 11:24:33.219384 17172 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskRFF58w/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/postgres -l logfile start
2026-08-07 11:24:36.221 UTC [28751] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 11:24:36.221 UTC [28751] LOG: listening on IPv4 address "127.16.197.20", port 35091
2026-08-07 11:24:36.225 UTC [28751] LOG: listening on Unix socket "/tmp/.s.PGSQL.35091"
2026-08-07 11:24:36.250 UTC [28754] LOG: database system was shut down at 2026-08-07 11:24:34 UTC
2026-08-07 11:24:36.255 UTC [28751] LOG: database system is ready to accept connections
I20260807 11:24:37.274748 17172 mini_postgres.cc:96] Postgres bound to 35091
2026-08-07 11:24:37.298 UTC [28762] FATAL: database "slave" does not exist
127.16.197.20:35091 - accepting connections
I20260807 11:24:37.299806 17172 mini_ranger.cc:161] Starting Ranger...
I20260807 11:24:37.335434 17172 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 11:24:37.962029 17172 mini_ranger.cc:87] Created ranger Postgres database
I20260807 11:24:37.962203 17172 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ranger-admin
2026-08-07 11:24:38,335 [I] DB FLAVOR :POSTGRES
2026-08-07 11:24:38,335 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 11:24:38,335 [I] Checking connection..
2026-08-07 11:24:38,336 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:35091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 11:24:38,594 [I] Checking connection passed.
2026-08-07 11:24:38,594 [I] --------- Verifying version history table ---------
2026-08-07 11:24:38,594 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:35091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 11:24:38,885 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 11:24:38,885 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:35091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 11:24:39,148 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 11:24:39,148 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 11:24:39,149 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:35091/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 11:24:39.394 UTC [28829] WARNING: there is no transaction in progress
2026-08-07 11:24:39,402 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 11:24:39,402 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:35091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 11:24:39,669 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 11:24:39,669 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 11:24:39,669 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:35091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 11:24:39,946 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:35091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 11:24:40,222 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:35091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-2w3w.c.gcp-upstream.internal','N') ;"
2026-08-07 11:24:40,475 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:35091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 11:24:40,759 [I] Table x_portal_user does not exist in database ranger
2026-08-07 11:24:40,759 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:35091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 11:24:41,024 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 11:24:41,024 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:35091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 11:24:41,274 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:35091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 11:24:41,546 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 11:24:41,546 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:35091/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 11:24:42.636 UTC [28942] WARNING: there is no transaction in progress
2026-08-07 11:24:42.655 UTC [28942] WARNING: there is no transaction in progress
2026-08-07 11:24:42.675 UTC [28942] WARNING: there is no transaction in progress
2026-08-07 11:24:42.694 UTC [28942] WARNING: there is no transaction in progress
2026-08-07 11:24:42.715 UTC [28942] WARNING: there is no transaction in progress
2026-08-07 11:24:42.736 UTC [28942] WARNING: there is no transaction in progress
2026-08-07 11:24:42.803 UTC [28942] WARNING: there is no transaction in progress
2026-08-07 11:24:42.811 UTC [28942] WARNING: there is no transaction in progress
2026-08-07 11:24:42.828 UTC [28942] WARNING: there is no transaction in progress
2026-08-07 11:24:42.839 UTC [28942] WARNING: there is no transaction in progress
2026-08-07 11:24:42.851 UTC [28942] WARNING: there is no transaction in progress
2026-08-07 11:24:42.867 UTC [28942] WARNING: there is no transaction in progress
2026-08-07 11:24:42.879 UTC [28942] WARNING: there is no transaction in progress
2026-08-07 11:24:42.886 UTC [28942] WARNING: there is no transaction in progress
2026-08-07 11:24:42.895 UTC [28942] WARNING: there is no transaction in progress
2026-08-07 11:24:42.902 UTC [28942] WARNING: there is no transaction in progress
2026-08-07 11:24:43,693 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 11:24:43,694 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:35091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 11:24:43,975 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 11:24:43,975 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:35091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 11:24:44,232 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 11:24:44,232 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:35091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 11:24:44,513 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 11:24:44,513 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:35091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 11:24:44,757 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:35091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 11:24:45,017 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:35091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-2w3w.c.gcp-upstream.internal';"
2026-08-07 11:24:45,263 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 11:24:45,263 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:35091/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 11:24:45,509 [I] DB_PATCHES have already been applied
I20260807 11:24:45.516254 17172 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ranger-admin:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskRFF58w/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 11:24:45.516361 17172 mini_ranger.cc:193] Using host: 127.16.197.20
I20260807 11:24:45.536828 17172 mini_ranger.cc:227] Ranger admin URL: http://127.16.197.20:38855
Aug 07, 2026 11:24:45 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 11:24:45 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 11:24:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 11:24:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 11:24:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 11:24:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 11:24:46 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 11:24:46.724180 17172 mini_ranger.cc:160] Time spent starting Ranger: real 9.424s user 0.003s sys 0.058s
Aug 07, 2026 11:25:09 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 11:25:09 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 11:25:09 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 11:25:09 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 11:25:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 11:25:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 11:25:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 11:25:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 11:25:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 11:25:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 11:25:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 11:25:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 11:25:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 11:25:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 11:25:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 11:25:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 11:25:09 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 11:25:10 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 11:25:10.759459 17172 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.16.197.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.16.197.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.16.197.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/rangerkms_127.16.197.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.16.197.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/rangerkms_127.16.197.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.16.197.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.16.197.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.16.197.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/HTTP_127.16.197.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.16.197.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/HTTP_127.16.197.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 11:25:10 dist-test-slave-2w3w krb5kdc[28717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101910, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 11:25:10.999552 17172 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 11:25:11.024480 17172 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 11:25:11.929803 17172 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 11:25:11.929987 17172 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ranger-kms
I20260807 11:25:11.930114 17172 mini_ranger_kms.cc:227] Using postgres at 127.16.197.20:35091
2026-08-07 11:25:12,823 [I] DB FLAVOR :POSTGRES
2026-08-07 11:25:12,824 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 11:25:12,824 [I] Checking connection
2026-08-07 11:25:12,824 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:35091/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 11:25:13,088 [I] connection success
2026-08-07 11:25:13,088 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 11:25:13,088 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 11:25:13,089 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:35091/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 11:25:13,414 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 11:25:13,415 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 11:25:13,415 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 11:25:13,415 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:35091/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 11:25:13,733 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 11:25:14.252614 17172 mini_ranger_kms.cc:310] Created kms service
I20260807 11:25:14.485270 17172 mini_ranger_kms.cc:326] Created kudu user
I20260807 11:25:14.530472 17172 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 11:25:14 dist-test-slave-2w3w krb5kdc[28717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101910, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.16.197.20@KRBTEST.COM
I20260807 11:25:14.940435 17172 mini_ranger_kms.cc:384] Added ranger policy
I20260807 11:25:14.941843 17172 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ranger-kms:/tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 11:25:14.941924 17172 mini_ranger_kms.cc:240] Using host: 127.16.197.20
I20260807 11:25:14.968680 17172 mini_ranger_kms.cc:276] Ranger KMS PID: 29132
I20260807 11:25:14.968894 17172 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.16.197.20:39959
Aug 07, 2026 11:25:16 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 11:25:16 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 11:25:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 11:25:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 11:25:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 11:25:16 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260807 11:25:17.430368 17172 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.431s user 0.172s sys 0.454s
I20260807 11:25:17.430734 17172 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 11:25:17.431083 17172 mini_ranger_kms.cc:401] 127.16.197.20:39959/kms/v1/keys
Aug 07 11:25:17 dist-test-slave-2w3w krb5kdc[28717](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786101917, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.197.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 07 11:25:18 dist-test-slave-2w3w krb5kdc[28717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101917, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.197.20@KRBTEST.COM for HTTP/127.16.197.20@KRBTEST.COM
Aug 07 11:25:18 dist-test-slave-2w3w krb5kdc[28717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101917, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.197.20@KRBTEST.COM for HTTP/127.16.197.20@KRBTEST.COM
Aug 07 11:25:21 dist-test-slave-2w3w krb5kdc[28717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:25:21.576298 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:39137
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:37473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:39137
--ranger_config_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.197.20:39959/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:25:22.254565 29187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:25:22.255026 29187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:25:22.255295 29187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:25:22.266294 29187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:25:22.266557 29187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:25:22.266748 29187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:25:22.266948 29187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:25:22.267156 29187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:25:22.282855 29187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:37473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/master-0/wal
--ranger_kms_url=127.16.197.20:39959/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:39137
--ranger_config_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:39137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:25:22.284636 29187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:25:22.286689 29187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:25:22.306763 29192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:25:22.306675 29193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:25:22.307375 29196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:25:22.308055 29187 server_base.cc:1034] running on GCE node
I20260807 11:25:22.309039 29187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:25:22.310734 29187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:25:22.312026 29187 hybrid_clock.cc:648] HybridClock initialized: now 1786101922311907 us; error 50 us; skew 500 ppm
I20260807 11:25:22.312537 29187 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:25:22 dist-test-slave-2w3w krb5kdc[28717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101922, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:25:22.317209 29187 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:25:22.319329 29187 webserver.cc:460] Webserver started at http://127.16.197.62:34407/ using document root <none> and password file <none>
I20260807 11:25:22.320134 29187 fs_manager.cc:359] Metadata directory not provided
I20260807 11:25:22.320372 29187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:25:22.320799 29187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 11:25:22 dist-test-slave-2w3w krb5kdc[28717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101922, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for HTTP/127.16.197.20@KRBTEST.COM
I20260807 11:25:22.396770 29187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "87f0add6e0a44519a85d4062f318850e"
format_stamp: "Formatted at 2026-08-07 11:25:22 on dist-test-slave-2w3w"
server_key: "8ee23dc9fa51352c7668fbccbd0b956d"
server_key_iv: "361e9752ad5cf84c3d8c50af1613943a"
server_key_version: "kuduclusterkey@0"
I20260807 11:25:22.397810 29187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "87f0add6e0a44519a85d4062f318850e"
format_stamp: "Formatted at 2026-08-07 11:25:22 on dist-test-slave-2w3w"
server_key: "8ee23dc9fa51352c7668fbccbd0b956d"
server_key_iv: "361e9752ad5cf84c3d8c50af1613943a"
server_key_version: "kuduclusterkey@0"
I20260807 11:25:22.404973 29187 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260807 11:25:22.425693 29203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:25:22.427359 29187 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 11:25:22.427620 29187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "87f0add6e0a44519a85d4062f318850e"
format_stamp: "Formatted at 2026-08-07 11:25:22 on dist-test-slave-2w3w"
server_key: "8ee23dc9fa51352c7668fbccbd0b956d"
server_key_iv: "361e9752ad5cf84c3d8c50af1613943a"
server_key_version: "kuduclusterkey@0"
I20260807 11:25:22.427939 29187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:25:22.473632 29187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:25:22.478977 29187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:25:22.479513 29187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:25:22.525730 29187 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:39137
I20260807 11:25:22.526033 29255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:39137 every 8 connection(s)
I20260807 11:25:22.527648 29187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:25:22.529174 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 29187
I20260807 11:25:22.529485 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:25:22.530567 17172 external_mini_cluster.cc:1428] Setting key a4c817e3d07b1f065c42d1e69721bf47
I20260807 11:25:22.534566 29256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 11:25:22 dist-test-slave-2w3w krb5kdc[28717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:25:22.553551 29256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87f0add6e0a44519a85d4062f318850e: Bootstrap starting.
I20260807 11:25:22.558620 29256 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87f0add6e0a44519a85d4062f318850e: Neither blocks nor log segments found. Creating new log.
I20260807 11:25:22.560567 29256 log.cc:824] T 00000000000000000000000000000000 P 87f0add6e0a44519a85d4062f318850e: Log is configured to *not* fsync() on all Append() calls
I20260807 11:25:22.561101 29259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:25:22.533297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42394 (local address 127.16.197.62:39137)
0807 11:25:22.535076 (+ 1779us) server_negotiation.cc:206] Beginning negotiation
0807 11:25:22.535087 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 11:25:22.535155 (+ 68us) server_negotiation.cc:407] Connection header received
0807 11:25:22.537765 (+ 2610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:25:22.537790 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:25:22.538189 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:25:22.538534 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:25:22.540598 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:25:22.542518 (+ 1920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:25:22.544671 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:25:22.545553 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:25:22.545685 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:25:22.551106 (+ 5421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:25:22.551118 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:25:22.551136 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:25:22.551205 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:25:22.555884 (+ 4679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:25:22.556669 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:25:22.556678 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:25:22.556685 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:25:22.556899 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:25:22.557375 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:25:22.557379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:25:22.557381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:25:22.558108 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:25:22.558633 (+ 525us) server_negotiation.cc:1035] Waiting for connection context
0807 11:25:22.559069 (+ 436us) server_negotiation.cc:299] Negotiation successful
0807 11:25:22.559609 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1490,"thread_start_us":1089,"threads_started":1}
I20260807 11:25:22.565392 29256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87f0add6e0a44519a85d4062f318850e: No bootstrap required, opened a new log
I20260807 11:25:22.577368 29256 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87f0add6e0a44519a85d4062f318850e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f0add6e0a44519a85d4062f318850e" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 39137 } }
I20260807 11:25:22.577953 29256 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87f0add6e0a44519a85d4062f318850e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:25:22.578166 29256 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87f0add6e0a44519a85d4062f318850e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87f0add6e0a44519a85d4062f318850e, State: Initialized, Role: FOLLOWER
I20260807 11:25:22.578888 29256 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87f0add6e0a44519a85d4062f318850e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f0add6e0a44519a85d4062f318850e" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 39137 } }
I20260807 11:25:22.579293 29256 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87f0add6e0a44519a85d4062f318850e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:25:22.579515 29256 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87f0add6e0a44519a85d4062f318850e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:25:22.579764 29256 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87f0add6e0a44519a85d4062f318850e [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:25:22.583012 29256 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87f0add6e0a44519a85d4062f318850e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f0add6e0a44519a85d4062f318850e" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 39137 } }
I20260807 11:25:22.583726 29256 leader_election.cc:302] T 00000000000000000000000000000000 P 87f0add6e0a44519a85d4062f318850e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87f0add6e0a44519a85d4062f318850e; no voters:
I20260807 11:25:22.584780 29256 leader_election.cc:288] T 00000000000000000000000000000000 P 87f0add6e0a44519a85d4062f318850e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:25:22.584971 29261 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87f0add6e0a44519a85d4062f318850e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:25:22.587060 29261 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87f0add6e0a44519a85d4062f318850e [term 1 LEADER]: Becoming Leader. State: Replica: 87f0add6e0a44519a85d4062f318850e, State: Running, Role: LEADER
I20260807 11:25:22.588383 29256 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87f0add6e0a44519a85d4062f318850e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:25:22.588157 29261 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87f0add6e0a44519a85d4062f318850e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f0add6e0a44519a85d4062f318850e" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 39137 } }
I20260807 11:25:22.594380 29262 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87f0add6e0a44519a85d4062f318850e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87f0add6e0a44519a85d4062f318850e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f0add6e0a44519a85d4062f318850e" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 39137 } } }
I20260807 11:25:22.604745 29262 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87f0add6e0a44519a85d4062f318850e [sys.catalog]: This master's current role is: LEADER
I20260807 11:25:22.594277 29263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87f0add6e0a44519a85d4062f318850e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87f0add6e0a44519a85d4062f318850e. Latest consensus state: current_term: 1 leader_uuid: "87f0add6e0a44519a85d4062f318850e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f0add6e0a44519a85d4062f318850e" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 39137 } } }
I20260807 11:25:22.605501 29263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87f0add6e0a44519a85d4062f318850e [sys.catalog]: This master's current role is: LEADER
I20260807 11:25:22.608911 29274 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:25:22.609570 29256 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 11:25:22.629163 29274 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:25:22.646036 29274 catalog_manager.cc:1347] Generated new cluster ID: 60f0ff9538724085be66502a57d12136
I20260807 11:25:22.646204 29274 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:25:22.669080 29274 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:25:22.671907 29274 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:25:22.697081 29274 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87f0add6e0a44519a85d4062f318850e: Generated new TSK 0
I20260807 11:25:22.698518 29274 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.197.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
I20260807 11:25:23.619194 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.197.20:39959/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:0
--local_ip_for_outbound_sockets=127.16.197.1
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:39137
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:37473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 11:25:23 dist-test-slave-2w3w krb5kdc[28717](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786101923, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 11:25:24.408597 29305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:25:24.409279 29305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:25:24.409593 29305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:25:24.423588 29305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:25:24.423780 29305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:25:24.424039 29305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:25:24.439344 29305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:37473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.16.197.20:39959/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:39137
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:25:24.440954 29305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:25:24.442680 29305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:25:24.472817 29316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:25:24.475181 29305 server_base.cc:1034] running on GCE node
W20260807 11:25:24.475653 29315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:25:24.479326 29318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:25:24.480227 29305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:25:24.481643 29305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:25:24.482949 29305 hybrid_clock.cc:648] HybridClock initialized: now 1786101924482885 us; error 53 us; skew 500 ppm
I20260807 11:25:24.483564 29305 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:25:24 dist-test-slave-2w3w krb5kdc[28717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101924, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:25:24.490293 29305 init.cc:377] Logged in from keytab as kudu/127.16.197.1@KRBTEST.COM (short username kudu)
I20260807 11:25:24.500988 29305 webserver.cc:460] Webserver started at http://127.16.197.1:38389/ using document root <none> and password file <none>
I20260807 11:25:24.502007 29305 fs_manager.cc:359] Metadata directory not provided
I20260807 11:25:24.502327 29305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:25:24.502841 29305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 11:25:24 dist-test-slave-2w3w krb5kdc[28717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101924, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for HTTP/127.16.197.20@KRBTEST.COM
I20260807 11:25:24.528968 29305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-0/data/instance:
uuid: "7beef5f45fea4af2ba40659ce815fdc6"
format_stamp: "Formatted at 2026-08-07 11:25:24 on dist-test-slave-2w3w"
server_key: "1e4cd1923be7164f7d572307e5c35fdb"
server_key_iv: "e2bc9816d3cba2a212ca5204c2bc3e6f"
server_key_version: "kuduclusterkey@0"
I20260807 11:25:24.530406 29305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance:
uuid: "7beef5f45fea4af2ba40659ce815fdc6"
format_stamp: "Formatted at 2026-08-07 11:25:24 on dist-test-slave-2w3w"
server_key: "1e4cd1923be7164f7d572307e5c35fdb"
server_key_iv: "e2bc9816d3cba2a212ca5204c2bc3e6f"
server_key_version: "kuduclusterkey@0"
I20260807 11:25:24.551756 29305 fs_manager.cc:691] Time spent creating directory manager: real 0.021s user 0.009s sys 0.013s
I20260807 11:25:24.591099 29325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:25:24.593680 29305 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.007s sys 0.007s
I20260807 11:25:24.594017 29305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "7beef5f45fea4af2ba40659ce815fdc6"
format_stamp: "Formatted at 2026-08-07 11:25:24 on dist-test-slave-2w3w"
server_key: "1e4cd1923be7164f7d572307e5c35fdb"
server_key_iv: "e2bc9816d3cba2a212ca5204c2bc3e6f"
server_key_version: "kuduclusterkey@0"
I20260807 11:25:24.594453 29305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 07 11:25:24 dist-test-slave-2w3w krb5kdc[28717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101923, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for HTTP/127.16.197.20@KRBTEST.COM
I20260807 11:25:24.676474 29305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:25:24.684719 29305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:25:24.685420 29305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:25:24.693854 29305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:25:24.703881 29305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:25:24.704176 29305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 11:25:24.704543 29305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:25:24.704748 29305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 11:25:25 dist-test-slave-2w3w krb5kdc[28717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101923, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for HTTP/127.16.197.20@KRBTEST.COM
I20260807 11:25:25.137940 29305 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:38391
I20260807 11:25:25.138016 29445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:38391 every 8 connection(s)
I20260807 11:25:25.139993 29305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:25:25.141642 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 29305
I20260807 11:25:25.141880 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance
I20260807 11:25:25.142807 17172 external_mini_cluster.cc:1428] Setting key 3466fbb811cd3c65577d092dcfe975f1
Aug 07 11:25:25 dist-test-slave-2w3w krb5kdc[28717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101924, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.197.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.2@KRBTEST.COM" created.
I20260807 11:25:25.256299 29448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:25:25.144618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:41993 (local address 127.16.197.62:39137)
0807 11:25:25.212944 (+ 68326us) server_negotiation.cc:206] Beginning negotiation
0807 11:25:25.212949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:25:25.212968 (+ 19us) server_negotiation.cc:407] Connection header received
0807 11:25:25.213024 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:25:25.213028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:25:25.213099 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:25:25.213255 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:25:25.216192 (+ 2937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:25:25.217579 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:25:25.220596 (+ 3017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:25:25.221587 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:25:25.221732 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:25:25.236676 (+ 14944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:25:25.236686 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:25:25.236691 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:25:25.236758 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:25:25.242131 (+ 5373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:25:25.248533 (+ 6402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:25:25.248542 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:25:25.248548 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:25:25.248749 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:25:25.249439 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:25:25.249447 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:25:25.249453 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:25:25.249943 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:25:25.250138 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 11:25:25.255578 (+ 5440us) server_negotiation.cc:299] Negotiation successful
0807 11:25:25.255886 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68189,"thread_start_us":68025,"threads_started":1}
I20260807 11:25:25.259452 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:25:25.145485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:39137 (local address 127.16.197.1:41993)
0807 11:25:25.200279 (+ 54794us) negotiation.cc:107] Waiting for socket to connect
0807 11:25:25.200366 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0807 11:25:25.209233 (+ 8867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:25:25.213590 (+ 4357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:25:25.213602 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:25:25.214516 (+ 914us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:25:25.215925 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:25:25.215957 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:25:25.217831 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:25:25.217837 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:25:25.220354 (+ 2517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:25:25.220385 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:25:25.222322 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:25:25.222332 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:25:25.222722 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:25:25.229437 (+ 6715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:25:25.229482 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:25:25.236154 (+ 6672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:25:25.247172 (+ 11018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:25:25.247188 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:25:25.247213 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:25:25.248306 (+ 1093us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:25:25.248908 (+ 602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:25:25.248916 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:25:25.248922 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:25:25.249269 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:25:25.250242 (+ 973us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:25:25.250255 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:25:25.250784 (+ 529us) client_negotiation.cc:769] Sending connection context
0807 11:25:25.256966 (+ 6182us) client_negotiation.cc:240] Negotiation successful
0807 11:25:25.257643 (+ 677us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54244,"thread_start_us":57340,"threads_started":1}
I20260807 11:25:25.262245 29446 heartbeater.cc:347] Connected to a master server at 127.16.197.62:39137
I20260807 11:25:25.262735 29446 heartbeater.cc:464] Registering TS with master...
I20260807 11:25:25.264135 29446 heartbeater.cc:511] Master 127.16.197.62:39137 requested a full tablet report, sending...
I20260807 11:25:25.267398 29220 ts_manager.cc:194] Registered new tserver with Master: 7beef5f45fea4af2ba40659ce815fdc6 (127.16.197.1:38391)
I20260807 11:25:25.270167 29220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.1@KRBTEST.COM'} at 127.16.197.1:41993
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
W20260807 11:25:25.290149 17172 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.16.197.2: real 0.140s user 0.000s sys 0.026s
I20260807 11:25:25.291244 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.197.20:39959/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:0
--local_ip_for_outbound_sockets=127.16.197.2
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:39137
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:37473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:25:25.950425 29452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:25:25.950834 29452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:25:25.951143 29452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:25:25.960741 29452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:25:25.960933 29452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:25:25.961164 29452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:25:25.974131 29452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:37473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.16.197.20:39959/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:39137
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:25:25.975800 29452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:25:25.977283 29452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:25:25.997323 29466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:25:25.997473 29467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:25:25.998358 29470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:25:25.999405 29452 server_base.cc:1034] running on GCE node
I20260807 11:25:26.000556 29452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:25:26.002298 29452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:25:26.003578 29452 hybrid_clock.cc:648] HybridClock initialized: now 1786101926003482 us; error 58 us; skew 500 ppm
I20260807 11:25:26.004025 29452 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:25:26 dist-test-slave-2w3w krb5kdc[28717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101926, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:25:26.008942 29452 init.cc:377] Logged in from keytab as kudu/127.16.197.2@KRBTEST.COM (short username kudu)
I20260807 11:25:26.011250 29452 webserver.cc:460] Webserver started at http://127.16.197.2:38565/ using document root <none> and password file <none>
I20260807 11:25:26.012145 29452 fs_manager.cc:359] Metadata directory not provided
I20260807 11:25:26.012310 29452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:25:26.012648 29452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 11:25:26 dist-test-slave-2w3w krb5kdc[28717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101926, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for HTTP/127.16.197.20@KRBTEST.COM
I20260807 11:25:26.029738 29452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-1/data/instance:
uuid: "44a923bb216a452b81fb9401198629a3"
format_stamp: "Formatted at 2026-08-07 11:25:26 on dist-test-slave-2w3w"
server_key: "ee7ff47e27d6ae964729422858d86b1d"
server_key_iv: "090321ff2274fffb4f2ea00720b4f987"
server_key_version: "kuduclusterkey@0"
I20260807 11:25:26.030764 29452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance:
uuid: "44a923bb216a452b81fb9401198629a3"
format_stamp: "Formatted at 2026-08-07 11:25:26 on dist-test-slave-2w3w"
server_key: "ee7ff47e27d6ae964729422858d86b1d"
server_key_iv: "090321ff2274fffb4f2ea00720b4f987"
server_key_version: "kuduclusterkey@0"
I20260807 11:25:26.038219 29452 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260807 11:25:26.057389 29477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:25:26.059329 29452 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 11:25:26.059605 29452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "44a923bb216a452b81fb9401198629a3"
format_stamp: "Formatted at 2026-08-07 11:25:26 on dist-test-slave-2w3w"
server_key: "ee7ff47e27d6ae964729422858d86b1d"
server_key_iv: "090321ff2274fffb4f2ea00720b4f987"
server_key_version: "kuduclusterkey@0"
I20260807 11:25:26.059952 29452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:25:26.079820 29452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:25:26.085227 29452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:25:26.085702 29452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:25:26.087352 29452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:25:26.089648 29452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:25:26.089830 29452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:25:26.090075 29452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:25:26.090193 29452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:25:26.180629 29452 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:43385
I20260807 11:25:26.180845 29598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:43385 every 8 connection(s)
I20260807 11:25:26.182423 29452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:25:26.191504 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 29452
I20260807 11:25:26.191735 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance
I20260807 11:25:26.192513 17172 external_mini_cluster.cc:1428] Setting key c455de540dfc84bc6d03680272f24137
Aug 07 11:25:26 dist-test-slave-2w3w krb5kdc[28717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101926, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:25:26.216245 29601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:25:26.186471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:50135 (local address 127.16.197.62:39137)
0807 11:25:26.187207 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0807 11:25:26.187212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:25:26.188508 (+ 1296us) server_negotiation.cc:407] Connection header received
0807 11:25:26.191358 (+ 2850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:25:26.191364 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:25:26.191467 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:25:26.191731 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:25:26.194306 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:25:26.195907 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:25:26.198154 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:25:26.199406 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:25:26.199609 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:25:26.204661 (+ 5052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:25:26.204679 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:25:26.204692 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:25:26.204761 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:25:26.211484 (+ 6723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:25:26.212487 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:25:26.212508 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:25:26.212514 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:25:26.212809 (+ 295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:25:26.213549 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:25:26.213557 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:25:26.213563 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:25:26.214065 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:25:26.214266 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0807 11:25:26.215118 (+ 852us) server_negotiation.cc:299] Negotiation successful
0807 11:25:26.215488 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":492,"threads_started":1}
I20260807 11:25:26.216714 29602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:25:26.187006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:39137 (local address 127.16.197.2:50135)
0807 11:25:26.188176 (+ 1170us) negotiation.cc:107] Waiting for socket to connect
0807 11:25:26.188243 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 11:25:26.191039 (+ 2796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:25:26.192255 (+ 1216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:25:26.192263 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:25:26.193003 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:25:26.194118 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:25:26.194143 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:25:26.196085 (+ 1942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:25:26.196089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:25:26.197932 (+ 1843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:25:26.197972 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:25:26.199734 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:25:26.199740 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:25:26.200068 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:25:26.201082 (+ 1014us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:25:26.201106 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:25:26.204277 (+ 3171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:25:26.211697 (+ 7420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:25:26.211705 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:25:26.211720 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:25:26.212282 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:25:26.213005 (+ 723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:25:26.213010 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:25:26.213012 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:25:26.213412 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:25:26.214219 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:25:26.214225 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:25:26.214588 (+ 363us) client_negotiation.cc:769] Sending connection context
0807 11:25:26.214922 (+ 334us) client_negotiation.cc:240] Negotiation successful
0807 11:25:26.215417 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":891,"thread_start_us":629,"threads_started":1}
I20260807 11:25:26.218835 29599 heartbeater.cc:347] Connected to a master server at 127.16.197.62:39137
I20260807 11:25:26.219430 29599 heartbeater.cc:464] Registering TS with master...
I20260807 11:25:26.221002 29599 heartbeater.cc:511] Master 127.16.197.62:39137 requested a full tablet report, sending...
I20260807 11:25:26.223230 29220 ts_manager.cc:194] Registered new tserver with Master: 44a923bb216a452b81fb9401198629a3 (127.16.197.2:43385)
I20260807 11:25:26.224758 29220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.2@KRBTEST.COM'} at 127.16.197.2:50135
WARNING: no policy specified for kudu/127.16.197.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.3@KRBTEST.COM" created.
I20260807 11:25:26.275115 29446 heartbeater.cc:503] Master 127.16.197.62:39137 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
I20260807 11:25:26.282718 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.197.20:39959/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:0
--local_ip_for_outbound_sockets=127.16.197.3
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:39137
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:37473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:25:26.904134 29605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:25:26.904516 29605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:25:26.904695 29605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:25:26.915692 29605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:25:26.915927 29605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:25:26.916203 29605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:25:26.931136 29605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:37473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.16.197.20:39959/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:39137
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:25:26.932813 29605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:25:26.934518 29605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:25:26.953706 29611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:25:26.953869 29610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:25:26.954052 29613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:25:26.955632 29605 server_base.cc:1034] running on GCE node
I20260807 11:25:26.956274 29605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:25:26.957247 29605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:25:26.958472 29605 hybrid_clock.cc:648] HybridClock initialized: now 1786101926958410 us; error 50 us; skew 500 ppm
I20260807 11:25:26.958786 29605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:25:26 dist-test-slave-2w3w krb5kdc[28717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101926, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:25:26.962419 29605 init.cc:377] Logged in from keytab as kudu/127.16.197.3@KRBTEST.COM (short username kudu)
I20260807 11:25:26.964795 29605 webserver.cc:460] Webserver started at http://127.16.197.3:45665/ using document root <none> and password file <none>
I20260807 11:25:26.965945 29605 fs_manager.cc:359] Metadata directory not provided
I20260807 11:25:26.966125 29605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:25:26.966475 29605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 11:25:26 dist-test-slave-2w3w krb5kdc[28717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101926, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for HTTP/127.16.197.20@KRBTEST.COM
I20260807 11:25:26.985129 29605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-2/data/instance:
uuid: "fac16b2efc88453d8563a213d4cbe5b4"
format_stamp: "Formatted at 2026-08-07 11:25:26 on dist-test-slave-2w3w"
server_key: "277f89ba49a8f3598572a291a48c9ca0"
server_key_iv: "d0955503888dfa3bd3d344dc4a9f71df"
server_key_version: "kuduclusterkey@0"
I20260807 11:25:26.986147 29605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance:
uuid: "fac16b2efc88453d8563a213d4cbe5b4"
format_stamp: "Formatted at 2026-08-07 11:25:26 on dist-test-slave-2w3w"
server_key: "277f89ba49a8f3598572a291a48c9ca0"
server_key_iv: "d0955503888dfa3bd3d344dc4a9f71df"
server_key_version: "kuduclusterkey@0"
I20260807 11:25:26.994257 29605 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.008s sys 0.001s
I20260807 11:25:27.014784 29621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:25:27.016752 29605 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 11:25:27.016992 29605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "fac16b2efc88453d8563a213d4cbe5b4"
format_stamp: "Formatted at 2026-08-07 11:25:26 on dist-test-slave-2w3w"
server_key: "277f89ba49a8f3598572a291a48c9ca0"
server_key_iv: "d0955503888dfa3bd3d344dc4a9f71df"
server_key_version: "kuduclusterkey@0"
I20260807 11:25:27.017302 29605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:25:27.036579 29605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:25:27.041838 29605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:25:27.042209 29605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:25:27.043972 29605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:25:27.049271 29605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:25:27.049413 29605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:25:27.049619 29605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:25:27.049721 29605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:25:27.134845 29605 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:45163
I20260807 11:25:27.134953 29741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:45163 every 8 connection(s)
I20260807 11:25:27.136500 29605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:25:27.140323 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 29605
I20260807 11:25:27.140547 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance
I20260807 11:25:27.141026 17172 external_mini_cluster.cc:1428] Setting key 0d55a3906382d973af5888bb8ea6b68a
Aug 07 11:25:27 dist-test-slave-2w3w krb5kdc[28717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101926, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:25:27.167501 29744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:25:27.140295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:59745 (local address 127.16.197.62:39137)
0807 11:25:27.144330 (+ 4035us) server_negotiation.cc:206] Beginning negotiation
0807 11:25:27.144337 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:25:27.144357 (+ 20us) server_negotiation.cc:407] Connection header received
0807 11:25:27.147739 (+ 3382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:25:27.147744 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:25:27.147915 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:25:27.148154 (+ 239us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:25:27.150128 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.151250 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.153257 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.153865 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.153982 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:25:27.159075 (+ 5093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:25:27.159082 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:25:27.159086 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:25:27.159124 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:25:27.163369 (+ 4245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:25:27.164280 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:25:27.164284 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:25:27.164286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:25:27.164452 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:25:27.165120 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:25:27.165124 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:25:27.165126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:25:27.165512 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:25:27.165724 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 11:25:27.166585 (+ 861us) server_negotiation.cc:299] Negotiation successful
0807 11:25:27.166843 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3882,"thread_start_us":3659,"threads_started":1}
I20260807 11:25:27.168792 29745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:25:27.141129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:39137 (local address 127.16.197.3:59745)
0807 11:25:27.142792 (+ 1663us) negotiation.cc:107] Waiting for socket to connect
0807 11:25:27.142836 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 11:25:27.147322 (+ 4486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:25:27.148380 (+ 1058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:25:27.148390 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:25:27.149070 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:25:27.149960 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:25:27.149980 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.151413 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.151416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:25:27.153128 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:25:27.153140 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.154143 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.154149 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:25:27.154434 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:25:27.155323 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:25:27.155347 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:25:27.158713 (+ 3366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:25:27.163521 (+ 4808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:25:27.163528 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:25:27.163546 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:25:27.164167 (+ 621us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:25:27.164549 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:25:27.164553 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:25:27.164555 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:25:27.164995 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:25:27.165657 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:25:27.165680 (+ 23us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:25:27.166093 (+ 413us) client_negotiation.cc:769] Sending connection context
0807 11:25:27.166458 (+ 365us) client_negotiation.cc:240] Negotiation successful
0807 11:25:27.167374 (+ 916us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1340,"thread_start_us":1008,"threads_started":1}
I20260807 11:25:27.170480 29742 heartbeater.cc:347] Connected to a master server at 127.16.197.62:39137
I20260807 11:25:27.170909 29742 heartbeater.cc:464] Registering TS with master...
I20260807 11:25:27.172032 29742 heartbeater.cc:511] Master 127.16.197.62:39137 requested a full tablet report, sending...
I20260807 11:25:27.173732 29220 ts_manager.cc:194] Registered new tserver with Master: fac16b2efc88453d8563a213d4cbe5b4 (127.16.197.3:45163)
I20260807 11:25:27.174830 29220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.3@KRBTEST.COM'} at 127.16.197.3:59745
I20260807 11:25:27.186571 17172 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 11:25:27.211419 29744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:25:27.193165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42400 (local address 127.16.197.62:39137)
0807 11:25:27.193336 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 11:25:27.193341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:25:27.194081 (+ 740us) server_negotiation.cc:407] Connection header received
0807 11:25:27.194325 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:25:27.194337 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:25:27.194415 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:25:27.194524 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:25:27.196836 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.197867 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.199891 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.200663 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.200834 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:25:27.204167 (+ 3333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:25:27.204173 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:25:27.204177 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:25:27.204221 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:25:27.207829 (+ 3608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:25:27.208734 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:25:27.208739 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:25:27.208743 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:25:27.208990 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:25:27.209465 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:25:27.209471 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:25:27.209474 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:25:27.209821 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:25:27.210253 (+ 432us) server_negotiation.cc:1035] Waiting for connection context
0807 11:25:27.210625 (+ 372us) server_negotiation.cc:299] Negotiation successful
0807 11:25:27.210992 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 11:25:27.216244 29220 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42400:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 11:25:27.227316 29220 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:25:27.228199 29599 heartbeater.cc:503] Master 127.16.197.62:39137 was elected leader, sending a full tablet report...
I20260807 11:25:27.266193 29756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:25:27.247139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55334 (local address 127.16.197.1:38391)
0807 11:25:27.248311 (+ 1172us) server_negotiation.cc:206] Beginning negotiation
0807 11:25:27.248322 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 11:25:27.251475 (+ 3153us) server_negotiation.cc:407] Connection header received
0807 11:25:27.251878 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:25:27.251883 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:25:27.252050 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:25:27.252330 (+ 280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:25:27.255092 (+ 2762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.256924 (+ 1832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.261757 (+ 4833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.263612 (+ 1855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.263791 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:25:27.264192 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0807 11:25:27.265295 (+ 1103us) server_negotiation.cc:299] Negotiation successful
0807 11:25:27.265514 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":929,"thread_start_us":794,"threads_started":1}
I20260807 11:25:27.266310 29758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:25:27.248246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:38391 (local address 127.0.0.1:55334)
0807 11:25:27.251358 (+ 3112us) negotiation.cc:107] Waiting for socket to connect
0807 11:25:27.251379 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 11:25:27.251640 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:25:27.252295 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:25:27.252300 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:25:27.252326 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:25:27.254683 (+ 2357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:25:27.254700 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.257081 (+ 2381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.257087 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:25:27.261471 (+ 4384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:25:27.261485 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.264705 (+ 3220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.264718 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:25:27.265009 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:25:27.265079 (+ 70us) client_negotiation.cc:769] Sending connection context
0807 11:25:27.265215 (+ 136us) client_negotiation.cc:240] Negotiation successful
0807 11:25:27.265604 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2875,"mutex_wait_us":50,"thread_start_us":2578,"threads_started":1}
I20260807 11:25:27.270027 29757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:25:27.246508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:43385 (local address 127.0.0.1:52210)
0807 11:25:27.249823 (+ 3315us) negotiation.cc:107] Waiting for socket to connect
0807 11:25:27.249865 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 11:25:27.250180 (+ 315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:25:27.251276 (+ 1096us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:25:27.251281 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:25:27.251348 (+ 67us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:25:27.258910 (+ 7562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:25:27.258941 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.261947 (+ 3006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.261955 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:25:27.265579 (+ 3624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:25:27.265594 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.268605 (+ 3011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.268616 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:25:27.268879 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:25:27.268936 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 11:25:27.269086 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 11:25:27.269220 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2994,"thread_start_us":2582,"threads_started":1}
I20260807 11:25:27.271338 29755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:25:27.246128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52210 (local address 127.16.197.2:43385)
0807 11:25:27.247922 (+ 1794us) server_negotiation.cc:206] Beginning negotiation
0807 11:25:27.247935 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 11:25:27.249975 (+ 2040us) server_negotiation.cc:407] Connection header received
0807 11:25:27.250501 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:25:27.250508 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:25:27.250699 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:25:27.250836 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:25:27.259248 (+ 8412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.261753 (+ 2505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.265819 (+ 4066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.268128 (+ 2309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.268329 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:25:27.268741 (+ 412us) server_negotiation.cc:1035] Waiting for connection context
0807 11:25:27.269176 (+ 435us) server_negotiation.cc:299] Negotiation successful
0807 11:25:27.269390 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1431,"thread_start_us":1125,"threads_started":1}
I20260807 11:25:27.271602 29376 tablet_service.cc:1467] Processing CreateTablet for tablet a580419a991f4af9bcd5dd841d04140c (DEFAULT_TABLE table=test-table [id=9b630f2e0e2b4a19b479cba7b4e95b8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:25:27.273437 29760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:25:27.251505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:45163 (local address 127.0.0.1:34012)
0807 11:25:27.258802 (+ 7297us) negotiation.cc:107] Waiting for socket to connect
0807 11:25:27.258824 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 11:25:27.259116 (+ 292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:25:27.259945 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:25:27.259950 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:25:27.259976 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:25:27.261002 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:25:27.261013 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.264219 (+ 3206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.264225 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:25:27.268617 (+ 4392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:25:27.268641 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.272297 (+ 3656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.272305 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:25:27.272567 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:25:27.272605 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 11:25:27.272727 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 11:25:27.272857 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6053,"thread_start_us":5741,"threads_started":1}
I20260807 11:25:27.273643 29376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a580419a991f4af9bcd5dd841d04140c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:25:27.273667 29759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:25:27.251999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34012 (local address 127.16.197.3:45163)
0807 11:25:27.253804 (+ 1805us) server_negotiation.cc:206] Beginning negotiation
0807 11:25:27.253814 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 11:25:27.259269 (+ 5455us) server_negotiation.cc:407] Connection header received
0807 11:25:27.259393 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:25:27.259400 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:25:27.259640 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:25:27.259961 (+ 321us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:25:27.261441 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.263990 (+ 2549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.268892 (+ 4902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.271383 (+ 2491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.271525 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:25:27.271947 (+ 422us) server_negotiation.cc:1035] Waiting for connection context
0807 11:25:27.272772 (+ 825us) server_negotiation.cc:299] Negotiation successful
0807 11:25:27.272987 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1569,"thread_start_us":1250,"threads_started":1}
I20260807 11:25:27.276268 29527 tablet_service.cc:1467] Processing CreateTablet for tablet a580419a991f4af9bcd5dd841d04140c (DEFAULT_TABLE table=test-table [id=9b630f2e0e2b4a19b479cba7b4e95b8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:25:27.277204 29670 tablet_service.cc:1467] Processing CreateTablet for tablet a580419a991f4af9bcd5dd841d04140c (DEFAULT_TABLE table=test-table [id=9b630f2e0e2b4a19b479cba7b4e95b8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:25:27.278846 29527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a580419a991f4af9bcd5dd841d04140c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:25:27.279161 29670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a580419a991f4af9bcd5dd841d04140c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:25:27.295367 29761 tablet_bootstrap.cc:492] T a580419a991f4af9bcd5dd841d04140c P fac16b2efc88453d8563a213d4cbe5b4: Bootstrap starting.
I20260807 11:25:27.300541 29762 tablet_bootstrap.cc:492] T a580419a991f4af9bcd5dd841d04140c P 7beef5f45fea4af2ba40659ce815fdc6: Bootstrap starting.
I20260807 11:25:27.301286 29761 tablet_bootstrap.cc:654] T a580419a991f4af9bcd5dd841d04140c P fac16b2efc88453d8563a213d4cbe5b4: Neither blocks nor log segments found. Creating new log.
I20260807 11:25:27.302709 29763 tablet_bootstrap.cc:492] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3: Bootstrap starting.
I20260807 11:25:27.303999 29761 log.cc:824] T a580419a991f4af9bcd5dd841d04140c P fac16b2efc88453d8563a213d4cbe5b4: Log is configured to *not* fsync() on all Append() calls
I20260807 11:25:27.306789 29762 tablet_bootstrap.cc:654] T a580419a991f4af9bcd5dd841d04140c P 7beef5f45fea4af2ba40659ce815fdc6: Neither blocks nor log segments found. Creating new log.
I20260807 11:25:27.308521 29763 tablet_bootstrap.cc:654] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3: Neither blocks nor log segments found. Creating new log.
I20260807 11:25:27.309551 29762 log.cc:824] T a580419a991f4af9bcd5dd841d04140c P 7beef5f45fea4af2ba40659ce815fdc6: Log is configured to *not* fsync() on all Append() calls
I20260807 11:25:27.310161 29761 tablet_bootstrap.cc:492] T a580419a991f4af9bcd5dd841d04140c P fac16b2efc88453d8563a213d4cbe5b4: No bootstrap required, opened a new log
I20260807 11:25:27.310590 29761 ts_tablet_manager.cc:1397] T a580419a991f4af9bcd5dd841d04140c P fac16b2efc88453d8563a213d4cbe5b4: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.000s
I20260807 11:25:27.310847 29763 log.cc:824] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3: Log is configured to *not* fsync() on all Append() calls
I20260807 11:25:27.317337 29762 tablet_bootstrap.cc:492] T a580419a991f4af9bcd5dd841d04140c P 7beef5f45fea4af2ba40659ce815fdc6: No bootstrap required, opened a new log
I20260807 11:25:27.317749 29762 ts_tablet_manager.cc:1397] T a580419a991f4af9bcd5dd841d04140c P 7beef5f45fea4af2ba40659ce815fdc6: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.011s
I20260807 11:25:27.318789 29763 tablet_bootstrap.cc:492] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3: No bootstrap required, opened a new log
I20260807 11:25:27.319259 29763 ts_tablet_manager.cc:1397] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.006s
I20260807 11:25:27.326570 29761 raft_consensus.cc:348] T a580419a991f4af9bcd5dd841d04140c P fac16b2efc88453d8563a213d4cbe5b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a923bb216a452b81fb9401198629a3" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 43385 } } peers { permanent_uuid: "7beef5f45fea4af2ba40659ce815fdc6" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38391 } } peers { permanent_uuid: "fac16b2efc88453d8563a213d4cbe5b4" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45163 } }
I20260807 11:25:27.327405 29761 raft_consensus.cc:374] T a580419a991f4af9bcd5dd841d04140c P fac16b2efc88453d8563a213d4cbe5b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:25:27.327587 29761 raft_consensus.cc:729] T a580419a991f4af9bcd5dd841d04140c P fac16b2efc88453d8563a213d4cbe5b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fac16b2efc88453d8563a213d4cbe5b4, State: Initialized, Role: FOLLOWER
I20260807 11:25:27.328671 29761 consensus_queue.cc:260] T a580419a991f4af9bcd5dd841d04140c P fac16b2efc88453d8563a213d4cbe5b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a923bb216a452b81fb9401198629a3" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 43385 } } peers { permanent_uuid: "7beef5f45fea4af2ba40659ce815fdc6" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38391 } } peers { permanent_uuid: "fac16b2efc88453d8563a213d4cbe5b4" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45163 } }
I20260807 11:25:27.331070 29742 heartbeater.cc:503] Master 127.16.197.62:39137 was elected leader, sending a full tablet report...
I20260807 11:25:27.332067 29761 ts_tablet_manager.cc:1428] T a580419a991f4af9bcd5dd841d04140c P fac16b2efc88453d8563a213d4cbe5b4: Time spent starting tablet: real 0.021s user 0.011s sys 0.013s
I20260807 11:25:27.332953 29763 raft_consensus.cc:348] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a923bb216a452b81fb9401198629a3" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 43385 } } peers { permanent_uuid: "7beef5f45fea4af2ba40659ce815fdc6" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38391 } } peers { permanent_uuid: "fac16b2efc88453d8563a213d4cbe5b4" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45163 } }
I20260807 11:25:27.333618 29763 raft_consensus.cc:374] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:25:27.333734 29763 raft_consensus.cc:729] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44a923bb216a452b81fb9401198629a3, State: Initialized, Role: FOLLOWER
I20260807 11:25:27.334472 29763 consensus_queue.cc:260] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a923bb216a452b81fb9401198629a3" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 43385 } } peers { permanent_uuid: "7beef5f45fea4af2ba40659ce815fdc6" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38391 } } peers { permanent_uuid: "fac16b2efc88453d8563a213d4cbe5b4" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45163 } }
I20260807 11:25:27.334945 29762 raft_consensus.cc:348] T a580419a991f4af9bcd5dd841d04140c P 7beef5f45fea4af2ba40659ce815fdc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a923bb216a452b81fb9401198629a3" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 43385 } } peers { permanent_uuid: "7beef5f45fea4af2ba40659ce815fdc6" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38391 } } peers { permanent_uuid: "fac16b2efc88453d8563a213d4cbe5b4" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45163 } }
I20260807 11:25:27.335567 29762 raft_consensus.cc:374] T a580419a991f4af9bcd5dd841d04140c P 7beef5f45fea4af2ba40659ce815fdc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:25:27.335719 29762 raft_consensus.cc:729] T a580419a991f4af9bcd5dd841d04140c P 7beef5f45fea4af2ba40659ce815fdc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7beef5f45fea4af2ba40659ce815fdc6, State: Initialized, Role: FOLLOWER
I20260807 11:25:27.336572 29762 consensus_queue.cc:260] T a580419a991f4af9bcd5dd841d04140c P 7beef5f45fea4af2ba40659ce815fdc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a923bb216a452b81fb9401198629a3" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 43385 } } peers { permanent_uuid: "7beef5f45fea4af2ba40659ce815fdc6" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38391 } } peers { permanent_uuid: "fac16b2efc88453d8563a213d4cbe5b4" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45163 } }
I20260807 11:25:27.337205 29763 ts_tablet_manager.cc:1428] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3: Time spent starting tablet: real 0.018s user 0.015s sys 0.001s
I20260807 11:25:27.339851 29762 ts_tablet_manager.cc:1428] T a580419a991f4af9bcd5dd841d04140c P 7beef5f45fea4af2ba40659ce815fdc6: Time spent starting tablet: real 0.022s user 0.021s sys 0.002s
W20260807 11:25:27.389427 29743 tablet.cc:2369] T a580419a991f4af9bcd5dd841d04140c P fac16b2efc88453d8563a213d4cbe5b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 11:25:27.436200 29600 tablet.cc:2369] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 11:25:27.464519 29447 tablet.cc:2369] T a580419a991f4af9bcd5dd841d04140c P 7beef5f45fea4af2ba40659ce815fdc6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:25:27.580843 29768 raft_consensus.cc:482] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:25:27.581133 29768 raft_consensus.cc:504] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a923bb216a452b81fb9401198629a3" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 43385 } } peers { permanent_uuid: "7beef5f45fea4af2ba40659ce815fdc6" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38391 } } peers { permanent_uuid: "fac16b2efc88453d8563a213d4cbe5b4" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45163 } }
I20260807 11:25:27.583652 29768 leader_election.cc:288] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7beef5f45fea4af2ba40659ce815fdc6 (127.16.197.1:38391), fac16b2efc88453d8563a213d4cbe5b4 (127.16.197.3:45163)
I20260807 11:25:27.593951 29756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:25:27.584532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:36649 (local address 127.16.197.1:38391)
0807 11:25:27.584942 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0807 11:25:27.584950 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:25:27.585403 (+ 453us) server_negotiation.cc:407] Connection header received
0807 11:25:27.585798 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:25:27.585805 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:25:27.585889 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:25:27.585964 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:25:27.587296 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.588351 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.591366 (+ 3015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.592609 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.592820 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:25:27.592942 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 11:25:27.593425 (+ 483us) server_negotiation.cc:299] Negotiation successful
0807 11:25:27.593566 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260807 11:25:27.594102 29770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:25:27.583993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:38391 (local address 127.16.197.2:36649)
0807 11:25:27.585245 (+ 1252us) negotiation.cc:107] Waiting for socket to connect
0807 11:25:27.585263 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 11:25:27.585554 (+ 291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:25:27.586182 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:25:27.586185 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:25:27.586207 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:25:27.587126 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:25:27.587144 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.588509 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.588517 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:25:27.591199 (+ 2682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:25:27.591212 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.592945 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.592954 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:25:27.593251 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:25:27.593277 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 11:25:27.593395 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 11:25:27.593550 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1101,"thread_start_us":814,"threads_started":1}
I20260807 11:25:27.595752 29397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a580419a991f4af9bcd5dd841d04140c" candidate_uuid: "44a923bb216a452b81fb9401198629a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7beef5f45fea4af2ba40659ce815fdc6" is_pre_election: true
I20260807 11:25:27.596165 29759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:25:27.585802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:47845 (local address 127.16.197.3:45163)
0807 11:25:27.586006 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 11:25:27.586012 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:25:27.587146 (+ 1134us) server_negotiation.cc:407] Connection header received
0807 11:25:27.587359 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:25:27.587365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:25:27.587437 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:25:27.587539 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:25:27.589106 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.590526 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.593294 (+ 2768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.594750 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.594996 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:25:27.595103 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 11:25:27.595629 (+ 526us) server_negotiation.cc:299] Negotiation successful
0807 11:25:27.595773 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 11:25:27.596397 29771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:25:27.585314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:45163 (local address 127.16.197.2:47845)
0807 11:25:27.587022 (+ 1708us) negotiation.cc:107] Waiting for socket to connect
0807 11:25:27.587048 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 11:25:27.587197 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:25:27.587563 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:25:27.587567 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:25:27.587593 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:25:27.588796 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:25:27.588809 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.590660 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.590666 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:25:27.593076 (+ 2410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:25:27.593089 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.595224 (+ 2135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.595232 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:25:27.595430 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:25:27.595455 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 11:25:27.595573 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 11:25:27.595774 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1497,"thread_start_us":1107,"threads_started":1}
I20260807 11:25:27.596735 29397 raft_consensus.cc:2454] T a580419a991f4af9bcd5dd841d04140c P 7beef5f45fea4af2ba40659ce815fdc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44a923bb216a452b81fb9401198629a3 in term 0.
I20260807 11:25:27.597826 29478 leader_election.cc:302] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44a923bb216a452b81fb9401198629a3, 7beef5f45fea4af2ba40659ce815fdc6; no voters:
I20260807 11:25:27.598099 29690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a580419a991f4af9bcd5dd841d04140c" candidate_uuid: "44a923bb216a452b81fb9401198629a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fac16b2efc88453d8563a213d4cbe5b4" is_pre_election: true
I20260807 11:25:27.598613 29768 raft_consensus.cc:2781] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:25:27.598832 29768 raft_consensus.cc:482] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:25:27.598953 29768 raft_consensus.cc:3037] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:25:27.598822 29690 raft_consensus.cc:2454] T a580419a991f4af9bcd5dd841d04140c P fac16b2efc88453d8563a213d4cbe5b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44a923bb216a452b81fb9401198629a3 in term 0.
I20260807 11:25:27.602514 29768 raft_consensus.cc:504] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a923bb216a452b81fb9401198629a3" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 43385 } } peers { permanent_uuid: "7beef5f45fea4af2ba40659ce815fdc6" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38391 } } peers { permanent_uuid: "fac16b2efc88453d8563a213d4cbe5b4" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45163 } }
I20260807 11:25:27.603434 29768 leader_election.cc:288] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3 [CANDIDATE]: Term 1 election: Requested vote from peers 7beef5f45fea4af2ba40659ce815fdc6 (127.16.197.1:38391), fac16b2efc88453d8563a213d4cbe5b4 (127.16.197.3:45163)
I20260807 11:25:27.604131 29397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a580419a991f4af9bcd5dd841d04140c" candidate_uuid: "44a923bb216a452b81fb9401198629a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7beef5f45fea4af2ba40659ce815fdc6"
I20260807 11:25:27.604197 29690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a580419a991f4af9bcd5dd841d04140c" candidate_uuid: "44a923bb216a452b81fb9401198629a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fac16b2efc88453d8563a213d4cbe5b4"
I20260807 11:25:27.604489 29690 raft_consensus.cc:3037] T a580419a991f4af9bcd5dd841d04140c P fac16b2efc88453d8563a213d4cbe5b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:25:27.604499 29397 raft_consensus.cc:3037] T a580419a991f4af9bcd5dd841d04140c P 7beef5f45fea4af2ba40659ce815fdc6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:25:27.610292 29397 raft_consensus.cc:2454] T a580419a991f4af9bcd5dd841d04140c P 7beef5f45fea4af2ba40659ce815fdc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44a923bb216a452b81fb9401198629a3 in term 1.
I20260807 11:25:27.610289 29690 raft_consensus.cc:2454] T a580419a991f4af9bcd5dd841d04140c P fac16b2efc88453d8563a213d4cbe5b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44a923bb216a452b81fb9401198629a3 in term 1.
I20260807 11:25:27.611047 29478 leader_election.cc:302] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44a923bb216a452b81fb9401198629a3, fac16b2efc88453d8563a213d4cbe5b4; no voters:
I20260807 11:25:27.611462 29768 raft_consensus.cc:2781] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:25:27.612442 29768 raft_consensus.cc:686] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3 [term 1 LEADER]: Becoming Leader. State: Replica: 44a923bb216a452b81fb9401198629a3, State: Running, Role: LEADER
I20260807 11:25:27.613296 29768 consensus_queue.cc:237] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a923bb216a452b81fb9401198629a3" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 43385 } } peers { permanent_uuid: "7beef5f45fea4af2ba40659ce815fdc6" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38391 } } peers { permanent_uuid: "fac16b2efc88453d8563a213d4cbe5b4" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45163 } }
I20260807 11:25:27.620002 29219 catalog_manager.cc:5495] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 44a923bb216a452b81fb9401198629a3 (127.16.197.2). New cstate: current_term: 1 leader_uuid: "44a923bb216a452b81fb9401198629a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44a923bb216a452b81fb9401198629a3" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 43385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7beef5f45fea4af2ba40659ce815fdc6" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fac16b2efc88453d8563a213d4cbe5b4" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45163 } health_report { overall_health: UNKNOWN } } }
I20260807 11:25:27.721079 29755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:25:27.713828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52226 (local address 127.16.197.2:43385)
0807 11:25:27.714194 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0807 11:25:27.714200 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:25:27.714570 (+ 370us) server_negotiation.cc:407] Connection header received
0807 11:25:27.714741 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:25:27.714747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:25:27.714842 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:25:27.715028 (+ 186us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:25:27.715652 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.716421 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.718603 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.719118 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.719265 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:25:27.719878 (+ 613us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:25:27.720458 (+ 580us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:25:27.720639 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 11:25:27.720696 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 11:25:27.720787 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260807 11:25:27.730433 29690 raft_consensus.cc:1261] T a580419a991f4af9bcd5dd841d04140c P fac16b2efc88453d8563a213d4cbe5b4 [term 1 FOLLOWER]: Refusing update from remote peer 44a923bb216a452b81fb9401198629a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:25:27.730683 29397 raft_consensus.cc:1261] T a580419a991f4af9bcd5dd841d04140c P 7beef5f45fea4af2ba40659ce815fdc6 [term 1 FOLLOWER]: Refusing update from remote peer 44a923bb216a452b81fb9401198629a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:25:27.731556 29772 consensus_queue.cc:1035] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fac16b2efc88453d8563a213d4cbe5b4" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:25:27.732054 29768 consensus_queue.cc:1035] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7beef5f45fea4af2ba40659ce815fdc6" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 38391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:25:27.770133 29219 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42400:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:25:27.770589 29219 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42400:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:25:27.775451 29219 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 87f0add6e0a44519a85d4062f318850e: Sending DeleteTablet for 3 replicas of tablet a580419a991f4af9bcd5dd841d04140c
I20260807 11:25:27.777019 29376 tablet_service.cc:1514] Processing DeleteTablet for tablet a580419a991f4af9bcd5dd841d04140c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:25:27 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:55334
I20260807 11:25:27.777033 29527 tablet_service.cc:1514] Processing DeleteTablet for tablet a580419a991f4af9bcd5dd841d04140c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:25:27 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:52210
I20260807 11:25:27.778390 29670 tablet_service.cc:1514] Processing DeleteTablet for tablet a580419a991f4af9bcd5dd841d04140c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:25:27 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:34012
I20260807 11:25:27.778648 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 29305
I20260807 11:25:27.792572 29786 tablet_replica.cc:330] T a580419a991f4af9bcd5dd841d04140c P 7beef5f45fea4af2ba40659ce815fdc6: stopping tablet replica
I20260807 11:25:27.793270 29786 raft_consensus.cc:2229] T a580419a991f4af9bcd5dd841d04140c P 7beef5f45fea4af2ba40659ce815fdc6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:25:27.793895 29786 raft_consensus.cc:2258] T a580419a991f4af9bcd5dd841d04140c P 7beef5f45fea4af2ba40659ce815fdc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:25:27.797196 29786 ts_tablet_manager.cc:1905] T a580419a991f4af9bcd5dd841d04140c P 7beef5f45fea4af2ba40659ce815fdc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:25:27.800364 29787 tablet_replica.cc:330] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3: stopping tablet replica
I20260807 11:25:27.800897 29787 raft_consensus.cc:2229] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3 [term 1 LEADER]: Raft consensus shutting down.
I20260807 11:25:27.801723 29787 raft_consensus.cc:2258] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:25:27.804527 29787 ts_tablet_manager.cc:1905] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 11:25:27 dist-test-slave-2w3w krb5kdc[28717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.1@KRBTEST.COM
I20260807 11:25:27.807902 29788 tablet_replica.cc:330] T a580419a991f4af9bcd5dd841d04140c P fac16b2efc88453d8563a213d4cbe5b4: stopping tablet replica
I20260807 11:25:27.808481 29788 raft_consensus.cc:2229] T a580419a991f4af9bcd5dd841d04140c P fac16b2efc88453d8563a213d4cbe5b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:25:27.809370 29788 raft_consensus.cc:2258] T a580419a991f4af9bcd5dd841d04140c P fac16b2efc88453d8563a213d4cbe5b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:25:27.810160 29786 ts_tablet_manager.cc:1918] T a580419a991f4af9bcd5dd841d04140c P 7beef5f45fea4af2ba40659ce815fdc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:25:27.810447 29786 log.cc:1196] T a580419a991f4af9bcd5dd841d04140c P 7beef5f45fea4af2ba40659ce815fdc6: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-0/wal/wals/a580419a991f4af9bcd5dd841d04140c
I20260807 11:25:27.811422 29786 ts_tablet_manager.cc:1939] T a580419a991f4af9bcd5dd841d04140c P 7beef5f45fea4af2ba40659ce815fdc6: Deleting consensus metadata
I20260807 11:25:27.815219 29788 ts_tablet_manager.cc:1905] T a580419a991f4af9bcd5dd841d04140c P fac16b2efc88453d8563a213d4cbe5b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:25:27.815397 29787 ts_tablet_manager.cc:1918] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:25:27.815748 29787 log.cc:1196] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-1/wal/wals/a580419a991f4af9bcd5dd841d04140c
I20260807 11:25:27.816587 29787 ts_tablet_manager.cc:1939] T a580419a991f4af9bcd5dd841d04140c P 44a923bb216a452b81fb9401198629a3: Deleting consensus metadata
I20260807 11:25:27.816885 29204 catalog_manager.cc:4841] TS 7beef5f45fea4af2ba40659ce815fdc6 (127.16.197.1:38391): tablet a580419a991f4af9bcd5dd841d04140c (table test-table [id=9b630f2e0e2b4a19b479cba7b4e95b8f]) successfully deleted
I20260807 11:25:27.818408 29205 catalog_manager.cc:4841] TS 44a923bb216a452b81fb9401198629a3 (127.16.197.2:43385): tablet a580419a991f4af9bcd5dd841d04140c (table test-table [id=9b630f2e0e2b4a19b479cba7b4e95b8f]) successfully deleted
I20260807 11:25:27.821964 29756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:25:27.779662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55346 (local address 127.16.197.1:38391)
0807 11:25:27.779917 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 11:25:27.779927 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 11:25:27.793802 (+ 13875us) server_negotiation.cc:407] Connection header received
0807 11:25:27.794365 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:25:27.794372 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:25:27.794465 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:25:27.794563 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:25:27.796826 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.798518 (+ 1692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.801018 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.801902 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.802006 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:25:27.809666 (+ 7660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:25:27.809684 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:25:27.809703 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:25:27.809793 (+ 90us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:25:27.816055 (+ 6262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:25:27.819158 (+ 3103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:25:27.819166 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:25:27.819168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:25:27.819356 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:25:27.819842 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:25:27.819846 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:25:27.819849 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:25:27.820340 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:25:27.820763 (+ 423us) server_negotiation.cc:1035] Waiting for connection context
0807 11:25:27.821118 (+ 355us) server_negotiation.cc:299] Negotiation successful
0807 11:25:27.821408 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 11:25:27.822897 29441 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:25:27.829037 29788 ts_tablet_manager.cc:1918] T a580419a991f4af9bcd5dd841d04140c P fac16b2efc88453d8563a213d4cbe5b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:25:27.829337 29788 log.cc:1196] T a580419a991f4af9bcd5dd841d04140c P fac16b2efc88453d8563a213d4cbe5b4: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786101765888975-17172-0/minicluster-data/ts-2/wal/wals/a580419a991f4af9bcd5dd841d04140c
I20260807 11:25:27.830159 29788 ts_tablet_manager.cc:1939] T a580419a991f4af9bcd5dd841d04140c P fac16b2efc88453d8563a213d4cbe5b4: Deleting consensus metadata
I20260807 11:25:27.832638 29204 catalog_manager.cc:4841] TS fac16b2efc88453d8563a213d4cbe5b4 (127.16.197.3:45163): tablet a580419a991f4af9bcd5dd841d04140c (table test-table [id=9b630f2e0e2b4a19b479cba7b4e95b8f]) successfully deleted
I20260807 11:25:27.944751 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 29305
I20260807 11:25:27.966428 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 29452
Aug 07 11:25:27 dist-test-slave-2w3w krb5kdc[28717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.2@KRBTEST.COM
I20260807 11:25:27.983361 29755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:25:27.967546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52232 (local address 127.16.197.2:43385)
0807 11:25:27.967762 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 11:25:27.967769 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:25:27.967794 (+ 25us) server_negotiation.cc:407] Connection header received
0807 11:25:27.967852 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:25:27.967856 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:25:27.967915 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:25:27.968045 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:25:27.969328 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.970159 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.971632 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:25:27.972089 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:25:27.972249 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:25:27.976742 (+ 4493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:25:27.976757 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:25:27.976773 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:25:27.976819 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:25:27.980274 (+ 3455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:25:27.980997 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:25:27.981011 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:25:27.981013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:25:27.981170 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:25:27.981587 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:25:27.981592 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:25:27.981602 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:25:27.982051 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:25:27.982370 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0807 11:25:27.982742 (+ 372us) server_negotiation.cc:299] Negotiation successful
0807 11:25:27.983015 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 11:25:27.984083 29594 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:25:28.110003 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 29452
I20260807 11:25:28.130482 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 29605
Aug 07 11:25:28 dist-test-slave-2w3w krb5kdc[28717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.3@KRBTEST.COM
I20260807 11:25:28.149241 29792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:25:28.131398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34028 (local address 127.16.197.3:45163)
0807 11:25:28.131902 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0807 11:25:28.131908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:25:28.131944 (+ 36us) server_negotiation.cc:407] Connection header received
0807 11:25:28.132004 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:25:28.132008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:25:28.132068 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:25:28.132169 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:25:28.133771 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:25:28.134591 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:25:28.136582 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:25:28.137023 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:25:28.137118 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:25:28.143047 (+ 5929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:25:28.143060 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:25:28.143075 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:25:28.143118 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:25:28.146242 (+ 3124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:25:28.147154 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:25:28.147162 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:25:28.147164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:25:28.147326 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:25:28.147707 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:25:28.147710 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:25:28.147712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:25:28.148067 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:25:28.148305 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0807 11:25:28.148631 (+ 326us) server_negotiation.cc:299] Negotiation successful
0807 11:25:28.148812 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":252,"threads_started":1}
I20260807 11:25:28.149972 29736 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:25:28.240656 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 29605
I20260807 11:25:28.259968 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 29187
I20260807 11:25:28.261226 29251 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:25:28.377206 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 29187
I20260807 11:25:28.404407 17172 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 11:25:28.750581 17172 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 11:25:28.751029 17172 mini_ranger.cc:66] Stopping Ranger...
I20260807 11:25:29.097157 17172 mini_ranger.cc:68] Stopped Ranger
2026-08-07 11:25:29.097 UTC [28751] LOG: received smart shutdown request
2026-08-07 11:25:29.101 UTC [28751] LOG: background worker "logical replication launcher" (PID 28760) exited with exit code 1
2026-08-07 11:25:29.101 UTC [28755] LOG: shutting down
2026-08-07 11:25:29.299 UTC [28751] LOG: database system is shut down
2026-08-07T11:25:29Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (57787 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:25:29 dist-test-slave-2w3w krb5kdc[29803](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:25:29 dist-test-slave-2w3w krb5kdc[29803](info): set up 2 sockets
Aug 07 11:25:29 dist-test-slave-2w3w krb5kdc[29803](info): commencing operation
krb5kdc: starting...
W20260807 11:25:30.600737 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:25:30 dist-test-slave-2w3w krb5kdc[29803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:25:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:25:30Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.16.197.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.16.197.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.16.197.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/rangeradmin_127.16.197.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.16.197.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/rangeradmin_127.16.197.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.16.197.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.16.197.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.16.197.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/rangerlookup_127.16.197.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.16.197.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/rangerlookup_127.16.197.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.16.197.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.16.197.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.16.197.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/HTTP_127.16.197.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.16.197.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/HTTP_127.16.197.20@KRBTEST.COM.keytab.
I20260807 11:25:30.976227 17172 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskRFF58w/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/postgres -l logfile start
2026-08-07 11:25:33.676 UTC [29836] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 11:25:33.676 UTC [29836] LOG: listening on IPv4 address "127.16.197.20", port 59057
2026-08-07 11:25:33.680 UTC [29836] LOG: listening on Unix socket "/tmp/.s.PGSQL.59057"
2026-08-07 11:25:33.706 UTC [29839] LOG: database system was shut down at 2026-08-07 11:25:32 UTC
2026-08-07 11:25:33.710 UTC [29836] LOG: database system is ready to accept connections
I20260807 11:25:34.720641 17172 mini_postgres.cc:96] Postgres bound to 59057
2026-08-07 11:25:34.741 UTC [29847] FATAL: database "slave" does not exist
127.16.197.20:59057 - accepting connections
I20260807 11:25:34.741998 17172 mini_ranger.cc:161] Starting Ranger...
I20260807 11:25:34.771725 17172 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 11:25:35.279776 17172 mini_ranger.cc:87] Created ranger Postgres database
I20260807 11:25:35.279915 17172 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ranger-admin
2026-08-07 11:25:35,597 [I] DB FLAVOR :POSTGRES
2026-08-07 11:25:35,597 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 11:25:35,597 [I] Checking connection..
2026-08-07 11:25:35,598 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:59057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 11:25:35,857 [I] Checking connection passed.
2026-08-07 11:25:35,858 [I] --------- Verifying version history table ---------
2026-08-07 11:25:35,858 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:59057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 11:25:36,096 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 11:25:36,097 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:59057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 11:25:36,331 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 11:25:36,332 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 11:25:36,332 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:59057/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 11:25:36.589 UTC [29915] WARNING: there is no transaction in progress
2026-08-07 11:25:36,597 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 11:25:36,597 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:59057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 11:25:36,864 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 11:25:36,864 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 11:25:36,865 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:59057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 11:25:37,136 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:59057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 11:25:37,411 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:59057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-2w3w.c.gcp-upstream.internal','N') ;"
2026-08-07 11:25:37,675 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:59057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 11:25:37,929 [I] Table x_portal_user does not exist in database ranger
2026-08-07 11:25:37,930 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:59057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 11:25:38,172 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 11:25:38,172 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:59057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 11:25:38,426 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:59057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 11:25:38,723 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 11:25:38,723 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:59057/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 11:25:39.747 UTC [30028] WARNING: there is no transaction in progress
2026-08-07 11:25:39.766 UTC [30028] WARNING: there is no transaction in progress
2026-08-07 11:25:39.783 UTC [30028] WARNING: there is no transaction in progress
2026-08-07 11:25:39.801 UTC [30028] WARNING: there is no transaction in progress
2026-08-07 11:25:39.818 UTC [30028] WARNING: there is no transaction in progress
2026-08-07 11:25:39.837 UTC [30028] WARNING: there is no transaction in progress
2026-08-07 11:25:39.896 UTC [30028] WARNING: there is no transaction in progress
2026-08-07 11:25:39.905 UTC [30028] WARNING: there is no transaction in progress
2026-08-07 11:25:39.924 UTC [30028] WARNING: there is no transaction in progress
2026-08-07 11:25:39.936 UTC [30028] WARNING: there is no transaction in progress
2026-08-07 11:25:39.947 UTC [30028] WARNING: there is no transaction in progress
2026-08-07 11:25:39.963 UTC [30028] WARNING: there is no transaction in progress
2026-08-07 11:25:39.975 UTC [30028] WARNING: there is no transaction in progress
2026-08-07 11:25:39.982 UTC [30028] WARNING: there is no transaction in progress
2026-08-07 11:25:39.989 UTC [30028] WARNING: there is no transaction in progress
2026-08-07 11:25:39.996 UTC [30028] WARNING: there is no transaction in progress
2026-08-07 11:25:40,717 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 11:25:40,718 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:59057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 11:25:40,968 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 11:25:40,968 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:59057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 11:25:41,241 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 11:25:41,242 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:59057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 11:25:41,512 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 11:25:41,513 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:59057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 11:25:41,775 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:59057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 11:25:42,013 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:59057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-2w3w.c.gcp-upstream.internal';"
2026-08-07 11:25:42,243 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 11:25:42,243 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:59057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 11:25:42,481 [I] DB_PATCHES have already been applied
I20260807 11:25:42.487179 17172 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ranger-admin:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskRFF58w/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 11:25:42.487270 17172 mini_ranger.cc:193] Using host: 127.16.197.20
I20260807 11:25:42.505489 17172 mini_ranger.cc:227] Ranger admin URL: http://127.16.197.20:43849
Aug 07, 2026 11:25:42 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 11:25:42 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 11:25:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 11:25:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 11:25:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 11:25:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 11:25:43 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 11:25:43.675205 17172 mini_ranger.cc:160] Time spent starting Ranger: real 8.933s user 0.003s sys 0.048s
Aug 07, 2026 11:26:03 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 11:26:03 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 11:26:03 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 11:26:03 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 11:26:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 11:26:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 11:26:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 11:26:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 11:26:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 11:26:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 11:26:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 11:26:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 11:26:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 11:26:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 11:26:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 11:26:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 11:26:03 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 11:26:04 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 11:26:05.207849 17172 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.16.197.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.16.197.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.16.197.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/rangerkms_127.16.197.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.16.197.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/rangerkms_127.16.197.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.16.197.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.16.197.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.16.197.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/HTTP_127.16.197.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.16.197.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/HTTP_127.16.197.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 11:26:05 dist-test-slave-2w3w krb5kdc[29803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101965, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 11:26:05.395061 17172 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 11:26:05.421116 17172 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 11:26:06.339777 17172 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 11:26:06.339908 17172 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ranger-kms
I20260807 11:26:06.339968 17172 mini_ranger_kms.cc:227] Using postgres at 127.16.197.20:59057
2026-08-07 11:26:06,708 [I] DB FLAVOR :POSTGRES
2026-08-07 11:26:06,709 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 11:26:06,709 [I] Checking connection
2026-08-07 11:26:06,709 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:59057/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 11:26:06,963 [I] connection success
2026-08-07 11:26:06,963 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 11:26:06,963 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 11:26:06,963 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:59057/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 11:26:07,231 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 11:26:07,231 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 11:26:07,231 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 11:26:07,231 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.197.20:59057/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 11:26:07,499 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 11:26:07.944410 17172 mini_ranger_kms.cc:310] Created kms service
I20260807 11:26:08.111505 17172 mini_ranger_kms.cc:326] Created kudu user
I20260807 11:26:08.158814 17172 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 11:26:08 dist-test-slave-2w3w krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101965, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.16.197.20@KRBTEST.COM
I20260807 11:26:08.564539 17172 mini_ranger_kms.cc:384] Added ranger policy
I20260807 11:26:08.564867 17172 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ranger-kms:/tmp/dist-test-taskRFF58w/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskRFF58w/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 11:26:08.564932 17172 mini_ranger_kms.cc:240] Using host: 127.16.197.20
I20260807 11:26:08.583950 17172 mini_ranger_kms.cc:276] Ranger KMS PID: 30214
I20260807 11:26:08.584092 17172 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.16.197.20:49443
Aug 07, 2026 11:26:09 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 11:26:09 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 11:26:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 11:26:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 11:26:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 11:26:10 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260807 11:26:11.020745 17172 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.626s user 0.068s sys 0.328s
I20260807 11:26:11.021037 17172 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 11:26:11.021243 17172 mini_ranger_kms.cc:401] 127.16.197.20:49443/kms/v1/keys
Aug 07 11:26:11 dist-test-slave-2w3w krb5kdc[29803](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786101971, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.197.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 07 11:26:11 dist-test-slave-2w3w krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101971, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.197.20@KRBTEST.COM for HTTP/127.16.197.20@KRBTEST.COM
Aug 07 11:26:11 dist-test-slave-2w3w krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101971, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.197.20@KRBTEST.COM for HTTP/127.16.197.20@KRBTEST.COM
Aug 07 11:26:14 dist-test-slave-2w3w krb5kdc[29803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:26:14.379637 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:33107
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:35799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:33107
--ranger_config_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.197.20:49443/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:26:14.732668 30269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:26:14.732982 30269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:26:14.733160 30269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:26:14.740609 30269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:26:14.740726 30269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:26:14.740824 30269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:26:14.740914 30269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:26:14.740986 30269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:26:14.751502 30269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:35799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/wal
--ranger_kms_url=127.16.197.20:49443/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:33107
--ranger_config_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:33107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:26:14.752892 30269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:26:14.754376 30269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:26:14.767140 30277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:14.767177 30275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:14.767210 30274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:14.767838 30269 server_base.cc:1034] running on GCE node
I20260807 11:26:14.768689 30269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:26:14.770246 30269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:26:14.771461 30269 hybrid_clock.cc:648] HybridClock initialized: now 1786101974771423 us; error 39 us; skew 500 ppm
I20260807 11:26:14.771853 30269 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:26:14 dist-test-slave-2w3w krb5kdc[29803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101974, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:26:14.775228 30269 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:26:14.776811 30269 webserver.cc:460] Webserver started at http://127.16.197.62:33863/ using document root <none> and password file <none>
I20260807 11:26:14.777475 30269 fs_manager.cc:359] Metadata directory not provided
I20260807 11:26:14.777602 30269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:26:14.777973 30269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 11:26:14 dist-test-slave-2w3w krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101974, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for HTTP/127.16.197.20@KRBTEST.COM
I20260807 11:26:14.838093 30269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "99c363da42cd45aeb31e929a95793c36"
format_stamp: "Formatted at 2026-08-07 11:26:14 on dist-test-slave-2w3w"
server_key: "1b20666db415aa6dbe79d528595cdad0"
server_key_iv: "f9e0b74571f17dc98bed544e6403ffaa"
server_key_version: "kuduclusterkey@0"
I20260807 11:26:14.839298 30269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "99c363da42cd45aeb31e929a95793c36"
format_stamp: "Formatted at 2026-08-07 11:26:14 on dist-test-slave-2w3w"
server_key: "1b20666db415aa6dbe79d528595cdad0"
server_key_iv: "f9e0b74571f17dc98bed544e6403ffaa"
server_key_version: "kuduclusterkey@0"
I20260807 11:26:14.846710 30269 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260807 11:26:14.865163 30284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:14.866760 30269 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260807 11:26:14.867045 30269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "99c363da42cd45aeb31e929a95793c36"
format_stamp: "Formatted at 2026-08-07 11:26:14 on dist-test-slave-2w3w"
server_key: "1b20666db415aa6dbe79d528595cdad0"
server_key_iv: "f9e0b74571f17dc98bed544e6403ffaa"
server_key_version: "kuduclusterkey@0"
I20260807 11:26:14.867318 30269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:26:14.895723 30269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:26:14.900264 30269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:26:14.900631 30269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:26:14.921623 30269 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:33107
I20260807 11:26:14.921627 30339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:33107 every 8 connection(s)
I20260807 11:26:14.923136 30269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:26:14.927218 30340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:26:14.929055 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 30269
I20260807 11:26:14.929293 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:26:14.930138 17172 external_mini_cluster.cc:1428] Setting key 310a4c479e3f80479453ff027376f0fa
I20260807 11:26:14.945041 30340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36: Bootstrap starting.
I20260807 11:26:14.949249 30340 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36: Neither blocks nor log segments found. Creating new log.
I20260807 11:26:14.950524 30340 log.cc:824] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36: Log is configured to *not* fsync() on all Append() calls
I20260807 11:26:14.954054 30340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36: No bootstrap required, opened a new log
Aug 07 11:26:14 dist-test-slave-2w3w krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:26:14.964279 30340 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99c363da42cd45aeb31e929a95793c36" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33107 } }
I20260807 11:26:14.964828 30340 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:26:14.964944 30340 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99c363da42cd45aeb31e929a95793c36, State: Initialized, Role: FOLLOWER
I20260807 11:26:14.965667 30340 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99c363da42cd45aeb31e929a95793c36" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33107 } }
I20260807 11:26:14.965934 30340 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:26:14.966099 30340 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:26:14.966323 30340 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:26:14.969369 30340 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99c363da42cd45aeb31e929a95793c36" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33107 } }
I20260807 11:26:14.969982 30340 leader_election.cc:302] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99c363da42cd45aeb31e929a95793c36; no voters:
I20260807 11:26:14.972533 30343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:14.932385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49046 (local address 127.16.197.62:33107)
0807 11:26:14.947162 (+ 14777us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:14.947179 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:14.947238 (+ 59us) server_negotiation.cc:407] Connection header received
0807 11:26:14.949766 (+ 2528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:14.949797 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:14.950346 (+ 549us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:14.950807 (+ 461us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:14.952583 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:14.954613 (+ 2030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:14.956618 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:14.957782 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:14.957972 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:14.963124 (+ 5152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:14.963137 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:14.963161 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:14.963220 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:14.967402 (+ 4182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:14.968374 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:14.968384 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:14.968398 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:14.968612 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:14.969080 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:14.969084 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:14.969087 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:14.969834 (+ 747us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:14.970161 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:14.970662 (+ 501us) server_negotiation.cc:299] Negotiation successful
0807 11:26:14.971162 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14472,"thread_start_us":16749,"threads_started":1}
I20260807 11:26:14.974457 30340 leader_election.cc:288] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:26:14.974551 30345 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:26:14.977099 30345 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [term 1 LEADER]: Becoming Leader. State: Replica: 99c363da42cd45aeb31e929a95793c36, State: Running, Role: LEADER
I20260807 11:26:14.977802 30345 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99c363da42cd45aeb31e929a95793c36" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33107 } }
I20260807 11:26:14.979828 30345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99c363da42cd45aeb31e929a95793c36. Latest consensus state: current_term: 1 leader_uuid: "99c363da42cd45aeb31e929a95793c36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99c363da42cd45aeb31e929a95793c36" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33107 } } }
I20260807 11:26:14.980232 30345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [sys.catalog]: This master's current role is: LEADER
I20260807 11:26:14.980739 30347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99c363da42cd45aeb31e929a95793c36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99c363da42cd45aeb31e929a95793c36" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33107 } } }
I20260807 11:26:14.981194 30347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [sys.catalog]: This master's current role is: LEADER
I20260807 11:26:14.983031 30340 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:26:14.987103 30349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:26:14.993259 30349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:26:15.015805 30349 catalog_manager.cc:1347] Generated new cluster ID: d0fb3ae98f904ee99cf6972a5ea66680
I20260807 11:26:15.015959 30349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:26:15.023269 30340 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 11:26:15.029983 30349 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:26:15.037086 30349 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:26:15.064965 30349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36: Generated new TSK 0
I20260807 11:26:15.066324 30349 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.197.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
I20260807 11:26:15.886821 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.197.20:49443/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:0
--local_ip_for_outbound_sockets=127.16.197.1
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:33107
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:35799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 11:26:16 dist-test-slave-2w3w krb5kdc[29803](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786101976, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 11:26:16.346172 30390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:26:16.346618 30390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:26:16.346832 30390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:26:16.357462 30390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:26:16.357627 30390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:26:16.357882 30390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:26:16.374099 30390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:35799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.16.197.20:49443/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:33107
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:26:16.375900 30390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:26:16.377600 30390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:26:16.395429 30404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:16.399712 30403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:16.400269 30390 server_base.cc:1034] running on GCE node
W20260807 11:26:16.400810 30406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:16.401731 30390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:26:16.403142 30390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:26:16.404440 30390 hybrid_clock.cc:648] HybridClock initialized: now 1786101976404379 us; error 46 us; skew 500 ppm
I20260807 11:26:16.404848 30390 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:26:16 dist-test-slave-2w3w krb5kdc[29803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101976, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:26:16.409350 30390 init.cc:377] Logged in from keytab as kudu/127.16.197.1@KRBTEST.COM (short username kudu)
I20260807 11:26:16.411423 30390 webserver.cc:460] Webserver started at http://127.16.197.1:36897/ using document root <none> and password file <none>
I20260807 11:26:16.412360 30390 fs_manager.cc:359] Metadata directory not provided
I20260807 11:26:16.412503 30390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:26:16.412850 30390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 11:26:16 dist-test-slave-2w3w krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101976, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for HTTP/127.16.197.20@KRBTEST.COM
I20260807 11:26:16.432538 30390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/data/instance:
uuid: "b66b68334ea547f19fe5ffc897229374"
format_stamp: "Formatted at 2026-08-07 11:26:16 on dist-test-slave-2w3w"
server_key: "6bac34201a3c9e3a01bdcf75b146f406"
server_key_iv: "ca23967d5c973c3af53bb8062af30218"
server_key_version: "kuduclusterkey@0"
I20260807 11:26:16.433650 30390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance:
uuid: "b66b68334ea547f19fe5ffc897229374"
format_stamp: "Formatted at 2026-08-07 11:26:16 on dist-test-slave-2w3w"
server_key: "6bac34201a3c9e3a01bdcf75b146f406"
server_key_iv: "ca23967d5c973c3af53bb8062af30218"
server_key_version: "kuduclusterkey@0"
I20260807 11:26:16.445925 30390 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.007s sys 0.003s
I20260807 11:26:16.467278 30415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:16.468905 30390 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 11:26:16.469116 30390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "b66b68334ea547f19fe5ffc897229374"
format_stamp: "Formatted at 2026-08-07 11:26:16 on dist-test-slave-2w3w"
server_key: "6bac34201a3c9e3a01bdcf75b146f406"
server_key_iv: "ca23967d5c973c3af53bb8062af30218"
server_key_version: "kuduclusterkey@0"
I20260807 11:26:16.469369 30390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:26:16.491050 30390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:26:16.496572 30390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:26:16.496992 30390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:26:16.504405 30390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:26:16.512669 30390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:26:16.512871 30390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 11:26:16.513077 30390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:26:16.513200 30390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:16.630767 30390 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:44821
I20260807 11:26:16.630829 30536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:44821 every 8 connection(s)
I20260807 11:26:16.632650 30390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:26:16.637109 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 30390
I20260807 11:26:16.637315 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance
I20260807 11:26:16.638171 17172 external_mini_cluster.cc:1428] Setting key 41861e0a3016b4102b97e55f9b6cde2c
Aug 07 11:26:16 dist-test-slave-2w3w krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101976, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:26:16.676713 30540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:16.647308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33107 (local address 127.16.197.1:57113)
0807 11:26:16.649780 (+ 2472us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:16.649845 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:16.652386 (+ 2541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:16.656095 (+ 3709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:16.656107 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:16.656776 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:26:16.657663 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:16.657693 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:16.659364 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:16.659369 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:16.661408 (+ 2039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:16.661424 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:16.662739 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:16.662753 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:16.663096 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:16.663972 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:26:16.664004 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:26:16.666707 (+ 2703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:26:16.671420 (+ 4713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:16.671431 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:16.671454 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:16.672125 (+ 671us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:16.672640 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:16.672645 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:16.672648 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:16.672951 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:16.673730 (+ 779us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:26:16.673739 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:26:16.674210 (+ 471us) client_negotiation.cc:769] Sending connection context
0807 11:26:16.674546 (+ 336us) client_negotiation.cc:240] Negotiation successful
0807 11:26:16.675170 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2132,"thread_start_us":7505,"threads_started":1}
I20260807 11:26:16.677949 30539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:16.647170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:57113 (local address 127.16.197.62:33107)
0807 11:26:16.655314 (+ 8144us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:16.655321 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:16.655344 (+ 23us) server_negotiation.cc:407] Connection header received
0807 11:26:16.655413 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:16.655419 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:16.655509 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:16.655624 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:16.657920 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:16.659148 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:16.661608 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:16.662266 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:16.662396 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:16.667048 (+ 4652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:16.667055 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:16.667060 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:16.667106 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:16.671144 (+ 4038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:16.672294 (+ 1150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:16.672300 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:16.672304 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:16.672495 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:16.673098 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:16.673104 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:16.673107 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:16.673493 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:16.673628 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:16.677261 (+ 3633us) server_negotiation.cc:299] Negotiation successful
0807 11:26:16.677573 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7857,"thread_start_us":7690,"threads_started":1}
I20260807 11:26:16.680290 30537 heartbeater.cc:347] Connected to a master server at 127.16.197.62:33107
I20260807 11:26:16.680666 30537 heartbeater.cc:464] Registering TS with master...
I20260807 11:26:16.681649 30537 heartbeater.cc:511] Master 127.16.197.62:33107 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.16.197.2@KRBTEST.COM; defaulting to no policy
I20260807 11:26:16.684268 30302 ts_manager.cc:194] Registered new tserver with Master: b66b68334ea547f19fe5ffc897229374 (127.16.197.1:44821)
I20260807 11:26:16.686285 30302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.1@KRBTEST.COM'} at 127.16.197.1:57113
Aug 07 11:26:16 dist-test-slave-2w3w krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101976, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for HTTP/127.16.197.20@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
I20260807 11:26:16.740180 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.197.20:49443/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:0
--local_ip_for_outbound_sockets=127.16.197.2
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:33107
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:35799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 11:26:16 dist-test-slave-2w3w krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101976, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for HTTP/127.16.197.20@KRBTEST.COM
W20260807 11:26:17.250181 30545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:26:17.250635 30545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:26:17.250810 30545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:26:17.262604 30545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:26:17.262794 30545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:26:17.263022 30545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:26:17.278743 30545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:35799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.16.197.20:49443/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:33107
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:26:17.280563 30545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:26:17.282416 30545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:26:17.298430 30560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:17.298693 30559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:17.299039 30562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:17.299424 30545 server_base.cc:1034] running on GCE node
I20260807 11:26:17.300073 30545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:26:17.301115 30545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:26:17.302366 30545 hybrid_clock.cc:648] HybridClock initialized: now 1786101977302286 us; error 57 us; skew 500 ppm
I20260807 11:26:17.302702 30545 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:26:17 dist-test-slave-2w3w krb5kdc[29803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101977, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:26:17.306236 30545 init.cc:377] Logged in from keytab as kudu/127.16.197.2@KRBTEST.COM (short username kudu)
I20260807 11:26:17.307982 30545 webserver.cc:460] Webserver started at http://127.16.197.2:43667/ using document root <none> and password file <none>
I20260807 11:26:17.308715 30545 fs_manager.cc:359] Metadata directory not provided
I20260807 11:26:17.308854 30545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:26:17.309214 30545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 11:26:17 dist-test-slave-2w3w krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101977, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for HTTP/127.16.197.20@KRBTEST.COM
I20260807 11:26:17.326316 30545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/data/instance:
uuid: "d34fc44d7aa0493bae4e72c5c23948ef"
format_stamp: "Formatted at 2026-08-07 11:26:17 on dist-test-slave-2w3w"
server_key: "99d049601465045be249e6c39786f746"
server_key_iv: "2e9dde0d434d78c47fc66d8855a6c6f3"
server_key_version: "kuduclusterkey@0"
I20260807 11:26:17.327597 30545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance:
uuid: "d34fc44d7aa0493bae4e72c5c23948ef"
format_stamp: "Formatted at 2026-08-07 11:26:17 on dist-test-slave-2w3w"
server_key: "99d049601465045be249e6c39786f746"
server_key_iv: "2e9dde0d434d78c47fc66d8855a6c6f3"
server_key_version: "kuduclusterkey@0"
I20260807 11:26:17.335420 30545 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260807 11:26:17.354892 30569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:17.356896 30545 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 11:26:17.357103 30545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "d34fc44d7aa0493bae4e72c5c23948ef"
format_stamp: "Formatted at 2026-08-07 11:26:17 on dist-test-slave-2w3w"
server_key: "99d049601465045be249e6c39786f746"
server_key_iv: "2e9dde0d434d78c47fc66d8855a6c6f3"
server_key_version: "kuduclusterkey@0"
I20260807 11:26:17.357460 30545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:26:17.403579 30545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:26:17.409253 30545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:26:17.409618 30545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:26:17.410895 30545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:26:17.412770 30545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:26:17.412875 30545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:17.413014 30545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:26:17.413079 30545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:17.461908 30545 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:38887
I20260807 11:26:17.461917 30687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:38887 every 8 connection(s)
I20260807 11:26:17.463773 30545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:26:17.469807 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 30545
I20260807 11:26:17.470012 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance
I20260807 11:26:17.470820 17172 external_mini_cluster.cc:1428] Setting key b3fa634a3e4f2e71c863cce9bdacdd6c
Aug 07 11:26:17 dist-test-slave-2w3w krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101977, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:26:17.502460 30691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:17.467491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33107 (local address 127.16.197.2:41665)
0807 11:26:17.471453 (+ 3962us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:17.471508 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:17.473968 (+ 2460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:17.480261 (+ 6293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:17.480276 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:17.481014 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:26:17.481828 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:17.481862 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:17.484472 (+ 2610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:17.484477 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:17.486591 (+ 2114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:17.486623 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:17.487919 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:17.487925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:17.488217 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:17.489094 (+ 877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:26:17.489125 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:26:17.492993 (+ 3868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:26:17.497394 (+ 4401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:17.497403 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:17.497422 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:17.498070 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:17.498542 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:17.498547 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:17.498551 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:17.498817 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:17.499614 (+ 797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:26:17.499623 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:26:17.500093 (+ 470us) client_negotiation.cc:769] Sending connection context
0807 11:26:17.500433 (+ 340us) client_negotiation.cc:240] Negotiation successful
0807 11:26:17.501030 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3636,"thread_start_us":10117,"threads_started":1}
I20260807 11:26:17.503684 30690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:17.466612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:41665 (local address 127.16.197.62:33107)
0807 11:26:17.479523 (+ 12911us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:17.479533 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:17.479564 (+ 31us) server_negotiation.cc:407] Connection header received
0807 11:26:17.479649 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:17.479658 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:17.479781 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:17.479924 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:17.483363 (+ 3439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:17.484291 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:17.486801 (+ 2510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:17.487479 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:17.487613 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:17.493277 (+ 5664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:17.493284 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:17.493288 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:17.493335 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:17.497188 (+ 3853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:17.498228 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:17.498234 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:17.498238 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:17.498395 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:17.498994 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:17.499000 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:17.499003 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:17.499376 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:17.499514 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:17.503053 (+ 3539us) server_negotiation.cc:299] Negotiation successful
0807 11:26:17.503313 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12725,"thread_start_us":12523,"threads_started":1}
I20260807 11:26:17.505816 30688 heartbeater.cc:347] Connected to a master server at 127.16.197.62:33107
I20260807 11:26:17.506171 30688 heartbeater.cc:464] Registering TS with master...
I20260807 11:26:17.507141 30688 heartbeater.cc:511] Master 127.16.197.62:33107 requested a full tablet report, sending...
I20260807 11:26:17.509074 30302 ts_manager.cc:194] Registered new tserver with Master: d34fc44d7aa0493bae4e72c5c23948ef (127.16.197.2:38887)
I20260807 11:26:17.510237 30302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.2@KRBTEST.COM'} at 127.16.197.2:41665
WARNING: no policy specified for kudu/127.16.197.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
I20260807 11:26:17.556074 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.197.20:49443/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:0
--local_ip_for_outbound_sockets=127.16.197.3
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:33107
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:35799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 11:26:17.690649 30537 heartbeater.cc:503] Master 127.16.197.62:33107 was elected leader, sending a full tablet report...
W20260807 11:26:17.934526 30694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:26:17.934850 30694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:26:17.934947 30694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:26:17.942520 30694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:26:17.942653 30694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:26:17.942776 30694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:26:17.953054 30694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:35799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.16.197.20:49443/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:33107
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:26:17.954447 30694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:26:17.955847 30694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:26:17.969784 30700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:17.969749 30701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:17.969657 30703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:17.970906 30694 server_base.cc:1034] running on GCE node
I20260807 11:26:17.971632 30694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:26:17.972700 30694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:26:17.973944 30694 hybrid_clock.cc:648] HybridClock initialized: now 1786101977973882 us; error 59 us; skew 500 ppm
I20260807 11:26:17.974282 30694 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:26:17 dist-test-slave-2w3w krb5kdc[29803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101977, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:26:17.977629 30694 init.cc:377] Logged in from keytab as kudu/127.16.197.3@KRBTEST.COM (short username kudu)
I20260807 11:26:17.979241 30694 webserver.cc:460] Webserver started at http://127.16.197.3:44801/ using document root <none> and password file <none>
I20260807 11:26:17.979980 30694 fs_manager.cc:359] Metadata directory not provided
I20260807 11:26:17.980118 30694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:26:17.980461 30694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 11:26:17 dist-test-slave-2w3w krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101977, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for HTTP/127.16.197.20@KRBTEST.COM
I20260807 11:26:17.998258 30694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/data/instance:
uuid: "512cff87a93e4ad1872d512d1ed7b9e7"
format_stamp: "Formatted at 2026-08-07 11:26:17 on dist-test-slave-2w3w"
server_key: "d34a36bba78b3eed5412e2f12226d9b3"
server_key_iv: "750a050039b35d97ae32c95740c3a556"
server_key_version: "kuduclusterkey@0"
I20260807 11:26:17.999552 30694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance:
uuid: "512cff87a93e4ad1872d512d1ed7b9e7"
format_stamp: "Formatted at 2026-08-07 11:26:17 on dist-test-slave-2w3w"
server_key: "d34a36bba78b3eed5412e2f12226d9b3"
server_key_iv: "750a050039b35d97ae32c95740c3a556"
server_key_version: "kuduclusterkey@0"
I20260807 11:26:18.005663 30694 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260807 11:26:18.022502 30712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:18.024508 30694 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 11:26:18.024705 30694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "512cff87a93e4ad1872d512d1ed7b9e7"
format_stamp: "Formatted at 2026-08-07 11:26:17 on dist-test-slave-2w3w"
server_key: "d34a36bba78b3eed5412e2f12226d9b3"
server_key_iv: "750a050039b35d97ae32c95740c3a556"
server_key_version: "kuduclusterkey@0"
I20260807 11:26:18.024987 30694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:26:18.045861 30694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:26:18.050865 30694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:26:18.051234 30694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:26:18.052444 30694 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:26:18.054236 30694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:26:18.054459 30694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:18.054617 30694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:26:18.054697 30694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:18.091068 30694 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:38697
I20260807 11:26:18.091128 30830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:38697 every 8 connection(s)
I20260807 11:26:18.092473 30694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:26:18.097378 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 30694
I20260807 11:26:18.097585 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance
I20260807 11:26:18.098147 17172 external_mini_cluster.cc:1428] Setting key f9601c918da114c77e38c8db080cf399
Aug 07 11:26:18 dist-test-slave-2w3w krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101977, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:26:18.113914 30833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:18.095129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:48335 (local address 127.16.197.62:33107)
0807 11:26:18.095614 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:18.095619 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:18.096592 (+ 973us) server_negotiation.cc:407] Connection header received
0807 11:26:18.099217 (+ 2625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:18.099223 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:18.099324 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:18.099454 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:18.101087 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:18.101993 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:18.103676 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:18.104125 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:18.104277 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:18.107711 (+ 3434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:18.107717 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:18.107720 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:18.107753 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:18.110315 (+ 2562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:18.111180 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:18.111185 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:18.111187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:18.111332 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:18.111734 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:18.111739 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:18.111741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:18.112153 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:18.112309 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:18.113254 (+ 945us) server_negotiation.cc:299] Negotiation successful
0807 11:26:18.113529 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":231,"threads_started":1}
I20260807 11:26:18.114931 30834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:18.095700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33107 (local address 127.16.197.3:48335)
0807 11:26:18.096419 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:18.096463 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:18.098888 (+ 2425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:18.099723 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:18.099735 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:18.100305 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:26:18.100911 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:18.100930 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:18.102147 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:18.102152 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:18.103512 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:18.103531 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:18.104369 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:18.104376 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:18.104616 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:18.105258 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:26:18.105284 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:26:18.107496 (+ 2212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:26:18.110471 (+ 2975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:18.110478 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:18.110492 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:18.111054 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:18.111431 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:18.111434 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:18.111436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:18.111614 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:18.112308 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:26:18.112314 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:26:18.112750 (+ 436us) client_negotiation.cc:769] Sending connection context
0807 11:26:18.113079 (+ 329us) client_negotiation.cc:240] Negotiation successful
0807 11:26:18.113595 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":299,"threads_started":1}
I20260807 11:26:18.116444 30831 heartbeater.cc:347] Connected to a master server at 127.16.197.62:33107
I20260807 11:26:18.116804 30831 heartbeater.cc:464] Registering TS with master...
I20260807 11:26:18.117560 30831 heartbeater.cc:511] Master 127.16.197.62:33107 requested a full tablet report, sending...
I20260807 11:26:18.119093 30302 ts_manager.cc:194] Registered new tserver with Master: 512cff87a93e4ad1872d512d1ed7b9e7 (127.16.197.3:38697)
I20260807 11:26:18.119930 30302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.3@KRBTEST.COM'} at 127.16.197.3:48335
I20260807 11:26:18.126919 17172 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 11:26:18.128741 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 30390
Aug 07 11:26:18 dist-test-slave-2w3w krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.1@KRBTEST.COM
I20260807 11:26:18.146901 30836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:18.129727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34754 (local address 127.16.197.1:44821)
0807 11:26:18.130365 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:18.130372 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:18.130400 (+ 28us) server_negotiation.cc:407] Connection header received
0807 11:26:18.130656 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:18.130663 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:18.130881 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:18.131109 (+ 228us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:18.132645 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:18.133698 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:18.135342 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:18.135913 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:18.136019 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:18.140603 (+ 4584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:18.140616 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:18.140631 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:18.140666 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:18.143679 (+ 3013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:18.144592 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:18.144608 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:18.144611 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:18.144773 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:18.145172 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:18.145178 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:18.145181 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:18.145539 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:18.145890 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:18.146231 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 11:26:18.146546 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":306,"threads_started":1}
I20260807 11:26:18.148285 30531 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:26:18.232458 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 30390
I20260807 11:26:18.250015 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 30545
Aug 07 11:26:18 dist-test-slave-2w3w krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.2@KRBTEST.COM
I20260807 11:26:18.266929 30838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:18.251026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60286 (local address 127.16.197.2:38887)
0807 11:26:18.251570 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:18.251577 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:18.251604 (+ 27us) server_negotiation.cc:407] Connection header received
0807 11:26:18.251682 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:18.251689 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:18.251902 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:18.252036 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:18.253281 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:18.254271 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:18.255800 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:18.256336 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:18.256434 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:18.260921 (+ 4487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:18.260933 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:18.260948 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:18.260987 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:18.264231 (+ 3244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:18.264946 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:18.264953 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:18.264955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:18.265089 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:18.265416 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:18.265419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:18.265421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:18.265767 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:18.265994 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:18.266292 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 11:26:18.266499 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":292,"threads_started":1}
I20260807 11:26:18.268282 30683 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:26:18.352255 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 30545
I20260807 11:26:18.374366 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 30694
Aug 07 11:26:18 dist-test-slave-2w3w krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.3@KRBTEST.COM
I20260807 11:26:18.392004 30840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:18.375554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49392 (local address 127.16.197.3:38697)
0807 11:26:18.376228 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:18.376235 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:18.376265 (+ 30us) server_negotiation.cc:407] Connection header received
0807 11:26:18.376322 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:18.376329 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:18.376548 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:18.376676 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:18.377921 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:18.379149 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:18.380458 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:18.381019 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:18.381121 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:18.385971 (+ 4850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:18.385983 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:18.385998 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:18.386038 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:18.389136 (+ 3098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:18.389898 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:18.389908 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:18.389911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:18.390062 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:18.390435 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:18.390438 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:18.390440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:18.390772 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:18.391093 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:18.391383 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 11:26:18.391585 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":373,"threads_started":1}
I20260807 11:26:18.393332 30826 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:26:18.478013 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 30694
I20260807 11:26:18.495744 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 30269
I20260807 11:26:18.497041 30334 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:26:18.607739 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 30269
I20260807 11:26:18.637189 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:33107
--webserver_interface=127.16.197.62
--webserver_port=33863
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:35799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:33107
--ranger_config_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.197.20:49443/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.16.197.20:49443/kms with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:26:18.980809 30843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:26:18.981190 30843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:26:18.981367 30843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:26:18.988833 30843 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:26:18.989015 30843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:26:18.989140 30843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:26:18.989246 30843 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:26:18.989343 30843 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:26:18.999995 30843 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:35799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.16.197.20:49443/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:33107
--ranger_config_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:33107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=33863
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:26:19.001474 30843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:26:19.002905 30843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:26:19.015661 30850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:19.015568 30851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:19.015637 30853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:19.016523 30843 server_base.cc:1034] running on GCE node
I20260807 11:26:19.017290 30843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:26:19.018765 30843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:26:19.019965 30843 hybrid_clock.cc:648] HybridClock initialized: now 1786101979019905 us; error 45 us; skew 500 ppm
I20260807 11:26:19.020288 30843 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:26:19 dist-test-slave-2w3w krb5kdc[29803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101979, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:26:19.023540 30843 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:26:19.025067 30843 webserver.cc:460] Webserver started at http://127.16.197.62:33863/ using document root <none> and password file <none>
I20260807 11:26:19.025738 30843 fs_manager.cc:359] Metadata directory not provided
I20260807 11:26:19.025866 30843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 11:26:19 dist-test-slave-2w3w krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101979, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for HTTP/127.16.197.20@KRBTEST.COM
I20260807 11:26:19.073417 30843 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 11:26:19.076807 30861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:19.078392 30843 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 11:26:19.078603 30843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "99c363da42cd45aeb31e929a95793c36"
format_stamp: "Formatted at 2026-08-07 11:26:14 on dist-test-slave-2w3w"
server_key: "1b20666db415aa6dbe79d528595cdad0"
server_key_iv: "f9e0b74571f17dc98bed544e6403ffaa"
server_key_version: "kuduclusterkey@0"
I20260807 11:26:19.079792 30843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:26:19.097219 30843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:26:19.101492 30843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:26:19.101807 30843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:26:19.121364 30843 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:33107
I20260807 11:26:19.121383 30916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:33107 every 8 connection(s)
I20260807 11:26:19.123003 30843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:26:19.125641 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 30843
I20260807 11:26:19.126513 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.197.20:49443/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:44821
--local_ip_for_outbound_sockets=127.16.197.1
--tserver_master_addrs=127.16.197.62:33107
--webserver_port=36897
--webserver_interface=127.16.197.1
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:35799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.16.197.20:49443/kms with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 11:26:19.130481 30917 sys_catalog.cc:263] Verifying existing consensus state
I20260807 11:26:19.134135 30917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36: Bootstrap starting.
I20260807 11:26:19.154693 30917 log.cc:824] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36: Log is configured to *not* fsync() on all Append() calls
I20260807 11:26:19.164485 30917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 11:26:19.165081 30917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36: Bootstrap complete.
I20260807 11:26:19.174409 30917 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99c363da42cd45aeb31e929a95793c36" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33107 } }
I20260807 11:26:19.175457 30917 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99c363da42cd45aeb31e929a95793c36, State: Initialized, Role: FOLLOWER
I20260807 11:26:19.176203 30917 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99c363da42cd45aeb31e929a95793c36" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33107 } }
I20260807 11:26:19.176445 30917 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:26:19.176570 30917 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:26:19.176779 30917 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [term 1 FOLLOWER]: Advancing to term 2
I20260807 11:26:19.180876 30917 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99c363da42cd45aeb31e929a95793c36" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33107 } }
I20260807 11:26:19.181413 30917 leader_election.cc:302] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99c363da42cd45aeb31e929a95793c36; no voters:
I20260807 11:26:19.182113 30917 leader_election.cc:288] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 11:26:19.182272 30922 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 11:26:19.183616 30922 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [term 2 LEADER]: Becoming Leader. State: Replica: 99c363da42cd45aeb31e929a95793c36, State: Running, Role: LEADER
I20260807 11:26:19.184369 30922 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99c363da42cd45aeb31e929a95793c36" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33107 } }
I20260807 11:26:19.184834 30917 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:26:19.187315 30923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "99c363da42cd45aeb31e929a95793c36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99c363da42cd45aeb31e929a95793c36" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33107 } } }
I20260807 11:26:19.187347 30924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99c363da42cd45aeb31e929a95793c36. Latest consensus state: current_term: 2 leader_uuid: "99c363da42cd45aeb31e929a95793c36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99c363da42cd45aeb31e929a95793c36" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33107 } } }
I20260807 11:26:19.187783 30924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [sys.catalog]: This master's current role is: LEADER
I20260807 11:26:19.187781 30923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36 [sys.catalog]: This master's current role is: LEADER
I20260807 11:26:19.189092 30928 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:26:19.197350 30928 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:26:19.198158 30917 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 11:26:19.199134 30928 catalog_manager.cc:1259] Loaded cluster ID: d0fb3ae98f904ee99cf6972a5ea66680
I20260807 11:26:19.199318 30928 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:26:19.203445 30928 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:26:19.205168 30928 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36: Loaded TSK: 0
I20260807 11:26:19.206997 30928 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 11:26:19.581796 30920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:26:19.582332 30920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:26:19.582571 30920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:26:19.594533 30920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:26:19.594769 30920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:26:19.595073 30920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:26:19.608896 30920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:35799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.16.197.20:49443/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:44821
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=36897
--tserver_master_addrs=127.16.197.62:33107
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:26:19.611027 30920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:26:19.612975 30920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:26:19.655529 30956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:19.655570 30955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:19.666462 30958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:19.667110 30920 server_base.cc:1034] running on GCE node
I20260807 11:26:19.668188 30920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:26:19.669687 30920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:26:19.671057 30920 hybrid_clock.cc:648] HybridClock initialized: now 1786101979670984 us; error 64 us; skew 500 ppm
I20260807 11:26:19.671520 30920 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:26:19 dist-test-slave-2w3w krb5kdc[29803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101979, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:26:19.676149 30920 init.cc:377] Logged in from keytab as kudu/127.16.197.1@KRBTEST.COM (short username kudu)
I20260807 11:26:19.680652 30920 webserver.cc:460] Webserver started at http://127.16.197.1:36897/ using document root <none> and password file <none>
I20260807 11:26:19.681746 30920 fs_manager.cc:359] Metadata directory not provided
I20260807 11:26:19.681929 30920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 11:26:19 dist-test-slave-2w3w krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101979, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for HTTP/127.16.197.20@KRBTEST.COM
I20260807 11:26:19.744084 30920 fs_manager.cc:709] Time spent opening directory manager: real 0.026s user 0.003s sys 0.015s
I20260807 11:26:19.758669 30966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:19.760916 30920 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.000s sys 0.011s
I20260807 11:26:19.761237 30920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "b66b68334ea547f19fe5ffc897229374"
format_stamp: "Formatted at 2026-08-07 11:26:16 on dist-test-slave-2w3w"
server_key: "6bac34201a3c9e3a01bdcf75b146f406"
server_key_iv: "ca23967d5c973c3af53bb8062af30218"
server_key_version: "kuduclusterkey@0"
I20260807 11:26:19.762805 30920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:26:19.802810 30920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:26:19.808823 30920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:26:19.809203 30920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:26:19.810678 30920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:26:19.812707 30920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:26:19.812839 30920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:19.812956 30920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:26:19.813042 30920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:19.856429 30920 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:44821
I20260807 11:26:19.856488 31086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:44821 every 8 connection(s)
I20260807 11:26:19.858482 30920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:26:19.860751 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 30920
I20260807 11:26:19.861613 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.197.20:49443/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:38887
--local_ip_for_outbound_sockets=127.16.197.2
--tserver_master_addrs=127.16.197.62:33107
--webserver_port=43667
--webserver_interface=127.16.197.2
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:35799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.16.197.20:49443/kms with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 11:26:19 dist-test-slave-2w3w krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101979, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:26:19.899082 31089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:19.862603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33107 (local address 127.16.197.1:52891)
0807 11:26:19.868440 (+ 5837us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:19.868492 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:19.870838 (+ 2346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:19.872505 (+ 1667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:19.872515 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:19.873100 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:26:19.873672 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:19.873690 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:19.877632 (+ 3942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:19.877641 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:19.880100 (+ 2459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:19.880122 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:19.882143 (+ 2021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:19.882153 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:19.882475 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:19.883520 (+ 1045us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:26:19.883554 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:26:19.886794 (+ 3240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:26:19.892124 (+ 5330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:19.892137 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:19.892161 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:19.892962 (+ 801us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:19.893681 (+ 719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:19.893690 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:19.893693 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:19.893976 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:19.895663 (+ 1687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:26:19.895672 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:26:19.896159 (+ 487us) client_negotiation.cc:769] Sending connection context
0807 11:26:19.896519 (+ 360us) client_negotiation.cc:240] Negotiation successful
0807 11:26:19.897164 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5489,"thread_start_us":5249,"threads_started":1}
I20260807 11:26:19.899204 31090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:19.862840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:52891 (local address 127.16.197.62:33107)
0807 11:26:19.867729 (+ 4889us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:19.867756 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:19.868651 (+ 895us) server_negotiation.cc:407] Connection header received
0807 11:26:19.871554 (+ 2903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:19.871568 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:19.871952 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:19.872353 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:19.873910 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:19.876987 (+ 3077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:19.880329 (+ 3342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:19.881553 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:19.881784 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:19.887252 (+ 5468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:19.887265 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:19.887287 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:19.887342 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:19.891881 (+ 4539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:19.893214 (+ 1333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:19.893230 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:19.893242 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:19.893513 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:19.894143 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:19.894152 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:19.894158 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:19.895093 (+ 935us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:19.895334 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:19.896846 (+ 1512us) server_negotiation.cc:299] Negotiation successful
0807 11:26:19.897334 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4527,"thread_start_us":4202,"threads_started":1}
I20260807 11:26:19.903148 31087 heartbeater.cc:347] Connected to a master server at 127.16.197.62:33107
I20260807 11:26:19.903466 31087 heartbeater.cc:464] Registering TS with master...
I20260807 11:26:19.904330 31087 heartbeater.cc:511] Master 127.16.197.62:33107 requested a full tablet report, sending...
W20260807 11:26:19.907852 31087 heartbeater.cc:644] Failed to heartbeat to 127.16.197.62:33107 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 11:26:19.908120 31087 heartbeater.cc:464] Registering TS with master...
I20260807 11:26:19.908456 31087 heartbeater.cc:511] Master 127.16.197.62:33107 requested a full tablet report, sending...
I20260807 11:26:19.910058 31087 heartbeater.cc:464] Registering TS with master...
I20260807 11:26:19.910490 31087 heartbeater.cc:511] Master 127.16.197.62:33107 requested a full tablet report, sending...
W20260807 11:26:19.911964 31087 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 11:26:20.296392 31091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:26:20.296685 31091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:26:20.296859 31091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:26:20.304451 31091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:26:20.304561 31091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:26:20.304767 31091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:26:20.314903 31091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:35799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.16.197.20:49443/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:38887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=43667
--tserver_master_addrs=127.16.197.62:33107
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:26:20.316628 31091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:26:20.318246 31091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:26:20.339304 31105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:20.339375 31106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:20.341531 31091 server_base.cc:1034] running on GCE node
W20260807 11:26:20.343312 31110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:20.344013 31091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:26:20.345165 31091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:26:20.346443 31091 hybrid_clock.cc:648] HybridClock initialized: now 1786101980346387 us; error 53 us; skew 500 ppm
I20260807 11:26:20.346786 31091 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:26:20 dist-test-slave-2w3w krb5kdc[29803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101980, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:26:20.351068 31091 init.cc:377] Logged in from keytab as kudu/127.16.197.2@KRBTEST.COM (short username kudu)
I20260807 11:26:20.360448 31091 webserver.cc:460] Webserver started at http://127.16.197.2:43667/ using document root <none> and password file <none>
I20260807 11:26:20.361323 31091 fs_manager.cc:359] Metadata directory not provided
I20260807 11:26:20.361502 31091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 11:26:20 dist-test-slave-2w3w krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101980, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for HTTP/127.16.197.20@KRBTEST.COM
I20260807 11:26:20.433116 31091 fs_manager.cc:709] Time spent opening directory manager: real 0.047s user 0.012s sys 0.028s
I20260807 11:26:20.447032 31119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:20.460181 31091 fs_manager.cc:725] Time spent opening block manager: real 0.025s user 0.007s sys 0.005s
I20260807 11:26:20.460506 31091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "d34fc44d7aa0493bae4e72c5c23948ef"
format_stamp: "Formatted at 2026-08-07 11:26:17 on dist-test-slave-2w3w"
server_key: "99d049601465045be249e6c39786f746"
server_key_iv: "2e9dde0d434d78c47fc66d8855a6c6f3"
server_key_version: "kuduclusterkey@0"
I20260807 11:26:20.462162 31091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:26:20.490991 31091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:26:20.496992 31091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:26:20.497323 31091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:26:20.498610 31091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:26:20.500419 31091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:26:20.500531 31091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:20.500635 31091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:26:20.500708 31091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 11:26:20 dist-test-slave-2w3w krb5kdc[29803](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786101980, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:26:20.608345 31091 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:38887
I20260807 11:26:20.608400 31241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:38887 every 8 connection(s)
I20260807 11:26:20.609956 31091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:26:20.618275 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 31091
I20260807 11:26:20.619131 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.197.20:49443/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:38697
--local_ip_for_outbound_sockets=127.16.197.3
--tserver_master_addrs=127.16.197.62:33107
--webserver_port=44801
--webserver_interface=127.16.197.3
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:35799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.16.197.20:49443/kms with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 11:26:20 dist-test-slave-2w3w krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101980, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:26:20.659879 31244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:20.616195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33107 (local address 127.16.197.2:38861)
0807 11:26:20.625170 (+ 8975us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:20.625247 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:20.629783 (+ 4536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:20.633938 (+ 4155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:20.633955 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:20.634689 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:26:20.635709 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:20.635742 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:20.637629 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:20.637638 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:20.640185 (+ 2547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:20.640205 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:20.641524 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:20.641534 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:20.641889 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:20.646482 (+ 4593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:26:20.646532 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:26:20.649812 (+ 3280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:26:20.654793 (+ 4981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:20.654805 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:20.654829 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:20.655606 (+ 777us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:20.656103 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:20.656108 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:20.656112 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:20.656367 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:20.657028 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:26:20.657036 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:26:20.657492 (+ 456us) client_negotiation.cc:769] Sending connection context
0807 11:26:20.657792 (+ 300us) client_negotiation.cc:240] Negotiation successful
0807 11:26:20.658383 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8554,"thread_start_us":9829,"threads_started":1}
I20260807 11:26:20.661840 31245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:20.616984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:38861 (local address 127.16.197.62:33107)
0807 11:26:20.633234 (+ 16250us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:20.633243 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:20.633269 (+ 26us) server_negotiation.cc:407] Connection header received
0807 11:26:20.633345 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:20.633353 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:20.633461 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:20.635927 (+ 2466us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:20.636141 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:20.637414 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:20.640441 (+ 3027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:20.641240 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:20.642430 (+ 1190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:20.650243 (+ 7813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:20.650253 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:20.650260 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:20.650313 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:20.654554 (+ 4241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:20.655810 (+ 1256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:20.655816 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:20.655820 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:20.655975 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:20.656511 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:20.656516 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:20.656519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:20.656880 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:20.661056 (+ 4176us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:20.661230 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 11:26:20.661476 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16069,"thread_start_us":25643,"threads_started":1}
I20260807 11:26:20.663599 31242 heartbeater.cc:347] Connected to a master server at 127.16.197.62:33107
I20260807 11:26:20.663930 31242 heartbeater.cc:464] Registering TS with master...
I20260807 11:26:20.664810 31242 heartbeater.cc:511] Master 127.16.197.62:33107 requested a full tablet report, sending...
I20260807 11:26:20.667250 30880 ts_manager.cc:194] Registered new tserver with Master: d34fc44d7aa0493bae4e72c5c23948ef (127.16.197.2:38887)
I20260807 11:26:20.669652 30880 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.2@KRBTEST.COM'} at 127.16.197.2:38861
I20260807 11:26:20.912113 31087 heartbeater.cc:464] Registering TS with master...
I20260807 11:26:20.912616 31087 heartbeater.cc:511] Master 127.16.197.62:33107 requested a full tablet report, sending...
I20260807 11:26:20.914166 30880 ts_manager.cc:194] Registered new tserver with Master: b66b68334ea547f19fe5ffc897229374 (127.16.197.1:44821)
I20260807 11:26:20.916863 30880 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.1@KRBTEST.COM'} at 127.16.197.1:52891
Aug 07 11:26:21 dist-test-slave-2w3w krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101980, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for HTTP/127.16.197.20@KRBTEST.COM
W20260807 11:26:21.102552 31246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:26:21.103036 31246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:26:21.103211 31246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:26:21.111709 31246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:26:21.111881 31246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:26:21.112076 31246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:26:21.123811 31246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:35799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.16.197.20:49443/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:38697
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=44801
--tserver_master_addrs=127.16.197.62:33107
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:26:21.125494 31246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:26:21.127264 31246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:26:21.143816 31251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:21.145017 31252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:21.153702 31246 server_base.cc:1034] running on GCE node
W20260807 11:26:21.153837 31254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:21.154753 31246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:26:21.155984 31246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:26:21.157223 31246 hybrid_clock.cc:648] HybridClock initialized: now 1786101981157177 us; error 81 us; skew 500 ppm
I20260807 11:26:21.157583 31246 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:26:21 dist-test-slave-2w3w krb5kdc[29803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101981, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:26:21.161232 31246 init.cc:377] Logged in from keytab as kudu/127.16.197.3@KRBTEST.COM (short username kudu)
I20260807 11:26:21.163343 31246 webserver.cc:460] Webserver started at http://127.16.197.3:44801/ using document root <none> and password file <none>
I20260807 11:26:21.164127 31246 fs_manager.cc:359] Metadata directory not provided
I20260807 11:26:21.164304 31246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 11:26:21 dist-test-slave-2w3w krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101981, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for HTTP/127.16.197.20@KRBTEST.COM
I20260807 11:26:21.189955 31246 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.001s
I20260807 11:26:21.193351 31264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:21.194923 31246 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 11:26:21.195206 31246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "512cff87a93e4ad1872d512d1ed7b9e7"
format_stamp: "Formatted at 2026-08-07 11:26:17 on dist-test-slave-2w3w"
server_key: "d34a36bba78b3eed5412e2f12226d9b3"
server_key_iv: "750a050039b35d97ae32c95740c3a556"
server_key_version: "kuduclusterkey@0"
I20260807 11:26:21.196285 31246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:26:21.219775 31246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:26:21.224120 31246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:26:21.224426 31246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:26:21.225571 31246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:26:21.227133 31246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:26:21.227252 31246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:21.227396 31246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:26:21.227486 31246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:21.301472 31246 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:38697
I20260807 11:26:21.301616 31383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:38697 every 8 connection(s)
I20260807 11:26:21.303333 31246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:26:21.309684 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 31246
Aug 07 11:26:21 dist-test-slave-2w3w krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101980, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for HTTP/127.16.197.20@KRBTEST.COM
Aug 07 11:26:21 dist-test-slave-2w3w krb5kdc[29803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101981, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:26:21.351733 31389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:21.310945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55464 (local address 127.16.197.62:33107)
0807 11:26:21.328861 (+ 17916us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:21.328868 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:21.328891 (+ 23us) server_negotiation.cc:407] Connection header received
0807 11:26:21.328974 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:21.328980 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:21.329064 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:21.329180 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:21.334608 (+ 5428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.335784 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.338525 (+ 2741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.339187 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.339317 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:21.343881 (+ 4564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:21.343887 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:21.343891 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:21.343945 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:21.347351 (+ 3406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:21.348895 (+ 1544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:21.348901 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:21.348904 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:21.349055 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:21.349790 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:21.349795 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:21.349798 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:21.350124 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:21.350390 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:21.351133 (+ 743us) server_negotiation.cc:299] Negotiation successful
0807 11:26:21.351365 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17773,"thread_start_us":22736,"threads_started":1}
I20260807 11:26:21.361545 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:21.320615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33107 (local address 127.16.197.3:57797)
0807 11:26:21.330353 (+ 9738us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:21.330415 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:21.333159 (+ 2744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:21.336045 (+ 2886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:21.336059 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:21.337499 (+ 1440us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:26:21.338332 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:21.338359 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.341045 (+ 2686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.341050 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:21.343061 (+ 2011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:21.343078 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.347496 (+ 4418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.347509 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:21.347844 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:21.348551 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:26:21.348580 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:26:21.352583 (+ 4003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:26:21.356592 (+ 4009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:21.356606 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:21.356626 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:21.357334 (+ 708us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:21.357772 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:21.357777 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:21.357781 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:21.358036 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:21.358732 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:26:21.358740 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:26:21.359213 (+ 473us) client_negotiation.cc:769] Sending connection context
0807 11:26:21.359521 (+ 308us) client_negotiation.cc:240] Negotiation successful
0807 11:26:21.360112 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9400,"thread_start_us":9136,"threads_started":1}
I20260807 11:26:21.362686 31386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:21.321337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:57797 (local address 127.16.197.62:33107)
0807 11:26:21.334003 (+ 12666us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:21.334010 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:21.334029 (+ 19us) server_negotiation.cc:407] Connection header received
0807 11:26:21.334096 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:21.334101 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:21.334188 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:21.339522 (+ 5334us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:21.339729 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.340862 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.343268 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.344822 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.348236 (+ 3414us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:21.352892 (+ 4656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:21.352898 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:21.352902 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:21.352941 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:21.356386 (+ 3445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:21.357488 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:21.357494 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:21.357497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:21.357637 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:21.358168 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:21.358172 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:21.358175 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:21.358522 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:21.358658 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:21.362111 (+ 3453us) server_negotiation.cc:299] Negotiation successful
0807 11:26:21.362340 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12529,"thread_start_us":7268,"threads_started":1}
I20260807 11:26:21.364105 31384 heartbeater.cc:347] Connected to a master server at 127.16.197.62:33107
I20260807 11:26:21.364459 31384 heartbeater.cc:464] Registering TS with master...
I20260807 11:26:21.365365 31384 heartbeater.cc:511] Master 127.16.197.62:33107 requested a full tablet report, sending...
I20260807 11:26:21.367218 30880 ts_manager.cc:194] Registered new tserver with Master: 512cff87a93e4ad1872d512d1ed7b9e7 (127.16.197.3:38697)
I20260807 11:26:21.368528 30880 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.3@KRBTEST.COM'} at 127.16.197.3:57797
I20260807 11:26:21.380662 17172 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 11:26:21.477116 31386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:21.425406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55466 (local address 127.16.197.62:33107)
0807 11:26:21.425624 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:21.425631 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:21.446152 (+ 20521us) server_negotiation.cc:407] Connection header received
0807 11:26:21.446746 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:21.446753 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:21.446849 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:21.446937 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:21.450464 (+ 3527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.451602 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.453697 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.454347 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.454456 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:21.469942 (+ 15486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:21.469949 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:21.469953 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:21.470001 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:21.473072 (+ 3071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:21.474073 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:21.474080 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:21.474084 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:21.474256 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:21.474814 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:21.474819 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:21.474821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:21.475211 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:21.475522 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:21.476523 (+ 1001us) server_negotiation.cc:299] Negotiation successful
0807 11:26:21.476771 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 11:26:21.483927 30880 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55466:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 11:26:21.494185 30880 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:26:21.610641 31405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:21.591037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:38697 (local address 127.0.0.1:49396)
0807 11:26:21.600710 (+ 9673us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:21.600733 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:21.600898 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:21.602464 (+ 1566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:21.602469 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:21.602496 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:26:21.602900 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:21.602913 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.604930 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.604936 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:21.607099 (+ 2163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:21.607116 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.609381 (+ 2265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.609388 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:21.609637 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:21.609711 (+ 74us) client_negotiation.cc:769] Sending connection context
0807 11:26:21.609855 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 11:26:21.610148 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9539,"mutex_wait_us":29,"thread_start_us":9944,"threads_started":1}
I20260807 11:26:21.611025 31402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:21.547894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55708 (local address 127.16.197.1:44821)
0807 11:26:21.581455 (+ 33561us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:21.581468 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:21.582494 (+ 1026us) server_negotiation.cc:407] Connection header received
0807 11:26:21.582623 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:21.582632 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:21.582918 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:21.585319 (+ 2401us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:26:21.586438 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.598422 (+ 11984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.607841 (+ 9419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.609342 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.610137 (+ 795us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:21.610391 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:21.610470 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 11:26:21.610632 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33348,"thread_start_us":33173,"threads_started":1}
I20260807 11:26:21.612267 31401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:21.547108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:44821 (local address 127.0.0.1:55708)
0807 11:26:21.579532 (+ 32424us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:21.579612 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:21.579848 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:21.585521 (+ 5673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:21.585528 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:21.585565 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:26:21.586012 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:21.586027 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.598643 (+ 12616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.598652 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:21.607550 (+ 8898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:21.607568 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.609686 (+ 2118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.609693 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:21.609880 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:21.609914 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 11:26:21.609980 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 11:26:21.611883 (+ 1903us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30623,"mutex_wait_us":1696,"thread_start_us":34597,"threads_started":1}
I20260807 11:26:21.613957 31400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:21.547634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:38887 (local address 127.0.0.1:41696)
0807 11:26:21.582061 (+ 34427us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:21.582083 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:21.582337 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:21.584254 (+ 1917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:21.584276 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:21.584320 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:26:21.584869 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:21.584888 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.593427 (+ 8539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.593436 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:21.596619 (+ 3183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:21.596644 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.613160 (+ 16516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.613168 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:21.613361 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:21.613398 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 11:26:21.613471 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 11:26:21.613623 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34224,"thread_start_us":32461,"threads_started":1}
I20260807 11:26:21.615288 31403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:21.548168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41696 (local address 127.16.197.2:38887)
0807 11:26:21.577145 (+ 28977us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:21.577159 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:21.582194 (+ 5035us) server_negotiation.cc:407] Connection header received
0807 11:26:21.583185 (+ 991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:21.583198 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:21.583533 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:21.583759 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:26:21.591026 (+ 7267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.593151 (+ 2125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.609921 (+ 16770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.612896 (+ 2975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.614321 (+ 1425us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:21.614585 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:21.614671 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 11:26:21.614876 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28728,"thread_start_us":29000,"threads_started":1}
I20260807 11:26:21.617264 31018 tablet_service.cc:1467] Processing CreateTablet for tablet 210aebcd7db5479c9dd5c4bd4c7e431c (DEFAULT_TABLE table=test-table [id=78c65b66b1464e7cb948459a6f1eefde]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:26:21.619356 31018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 210aebcd7db5479c9dd5c4bd4c7e431c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:26:21.619570 31171 tablet_service.cc:1467] Processing CreateTablet for tablet 210aebcd7db5479c9dd5c4bd4c7e431c (DEFAULT_TABLE table=test-table [id=78c65b66b1464e7cb948459a6f1eefde]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:26:21.621133 31171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 210aebcd7db5479c9dd5c4bd4c7e431c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:26:21.621603 31404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:21.582667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49396 (local address 127.16.197.3:38697)
0807 11:26:21.600205 (+ 17538us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:21.600214 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:21.601385 (+ 1171us) server_negotiation.cc:407] Connection header received
0807 11:26:21.601502 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:21.601511 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:21.601816 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:21.602035 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:26:21.603176 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.604746 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.607441 (+ 2695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.609115 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.620672 (+ 11557us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:21.620952 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:21.621049 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 11:26:21.621243 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17316,"thread_start_us":18462,"threads_started":1}
I20260807 11:26:21.625450 31316 tablet_service.cc:1467] Processing CreateTablet for tablet 210aebcd7db5479c9dd5c4bd4c7e431c (DEFAULT_TABLE table=test-table [id=78c65b66b1464e7cb948459a6f1eefde]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:26:21.626925 31316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 210aebcd7db5479c9dd5c4bd4c7e431c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:26:21.641433 31409 tablet_bootstrap.cc:492] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374: Bootstrap starting.
I20260807 11:26:21.648480 31409 tablet_bootstrap.cc:654] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374: Neither blocks nor log segments found. Creating new log.
I20260807 11:26:21.648543 31410 tablet_bootstrap.cc:492] T 210aebcd7db5479c9dd5c4bd4c7e431c P d34fc44d7aa0493bae4e72c5c23948ef: Bootstrap starting.
I20260807 11:26:21.650089 31409 log.cc:824] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374: Log is configured to *not* fsync() on all Append() calls
I20260807 11:26:21.653738 31413 tablet_bootstrap.cc:492] T 210aebcd7db5479c9dd5c4bd4c7e431c P 512cff87a93e4ad1872d512d1ed7b9e7: Bootstrap starting.
I20260807 11:26:21.655809 31410 tablet_bootstrap.cc:654] T 210aebcd7db5479c9dd5c4bd4c7e431c P d34fc44d7aa0493bae4e72c5c23948ef: Neither blocks nor log segments found. Creating new log.
I20260807 11:26:21.657106 31410 log.cc:824] T 210aebcd7db5479c9dd5c4bd4c7e431c P d34fc44d7aa0493bae4e72c5c23948ef: Log is configured to *not* fsync() on all Append() calls
I20260807 11:26:21.658332 31413 tablet_bootstrap.cc:654] T 210aebcd7db5479c9dd5c4bd4c7e431c P 512cff87a93e4ad1872d512d1ed7b9e7: Neither blocks nor log segments found. Creating new log.
I20260807 11:26:21.660297 31409 tablet_bootstrap.cc:492] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374: No bootstrap required, opened a new log
I20260807 11:26:21.660655 31409 ts_tablet_manager.cc:1397] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374: Time spent bootstrapping tablet: real 0.019s user 0.009s sys 0.007s
I20260807 11:26:21.661319 31413 log.cc:824] T 210aebcd7db5479c9dd5c4bd4c7e431c P 512cff87a93e4ad1872d512d1ed7b9e7: Log is configured to *not* fsync() on all Append() calls
I20260807 11:26:21.674536 31242 heartbeater.cc:503] Master 127.16.197.62:33107 was elected leader, sending a full tablet report...
I20260807 11:26:21.679589 31410 tablet_bootstrap.cc:492] T 210aebcd7db5479c9dd5c4bd4c7e431c P d34fc44d7aa0493bae4e72c5c23948ef: No bootstrap required, opened a new log
I20260807 11:26:21.680033 31410 ts_tablet_manager.cc:1397] T 210aebcd7db5479c9dd5c4bd4c7e431c P d34fc44d7aa0493bae4e72c5c23948ef: Time spent bootstrapping tablet: real 0.032s user 0.007s sys 0.019s
I20260807 11:26:21.680347 31409 raft_consensus.cc:348] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66b68334ea547f19fe5ffc897229374" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 44821 } } peers { permanent_uuid: "512cff87a93e4ad1872d512d1ed7b9e7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 38697 } } peers { permanent_uuid: "d34fc44d7aa0493bae4e72c5c23948ef" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38887 } }
I20260807 11:26:21.680948 31409 raft_consensus.cc:374] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:26:21.681129 31413 tablet_bootstrap.cc:492] T 210aebcd7db5479c9dd5c4bd4c7e431c P 512cff87a93e4ad1872d512d1ed7b9e7: No bootstrap required, opened a new log
I20260807 11:26:21.681063 31409 raft_consensus.cc:729] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b66b68334ea547f19fe5ffc897229374, State: Initialized, Role: FOLLOWER
I20260807 11:26:21.681427 31413 ts_tablet_manager.cc:1397] T 210aebcd7db5479c9dd5c4bd4c7e431c P 512cff87a93e4ad1872d512d1ed7b9e7: Time spent bootstrapping tablet: real 0.028s user 0.000s sys 0.022s
I20260807 11:26:21.682004 31409 consensus_queue.cc:260] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66b68334ea547f19fe5ffc897229374" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 44821 } } peers { permanent_uuid: "512cff87a93e4ad1872d512d1ed7b9e7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 38697 } } peers { permanent_uuid: "d34fc44d7aa0493bae4e72c5c23948ef" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38887 } }
I20260807 11:26:21.692094 31413 raft_consensus.cc:348] T 210aebcd7db5479c9dd5c4bd4c7e431c P 512cff87a93e4ad1872d512d1ed7b9e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66b68334ea547f19fe5ffc897229374" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 44821 } } peers { permanent_uuid: "512cff87a93e4ad1872d512d1ed7b9e7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 38697 } } peers { permanent_uuid: "d34fc44d7aa0493bae4e72c5c23948ef" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38887 } }
I20260807 11:26:21.692659 31413 raft_consensus.cc:374] T 210aebcd7db5479c9dd5c4bd4c7e431c P 512cff87a93e4ad1872d512d1ed7b9e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:26:21.692771 31413 raft_consensus.cc:729] T 210aebcd7db5479c9dd5c4bd4c7e431c P 512cff87a93e4ad1872d512d1ed7b9e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 512cff87a93e4ad1872d512d1ed7b9e7, State: Initialized, Role: FOLLOWER
I20260807 11:26:21.692782 31409 ts_tablet_manager.cc:1428] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374: Time spent starting tablet: real 0.032s user 0.016s sys 0.008s
I20260807 11:26:21.693326 31087 heartbeater.cc:503] Master 127.16.197.62:33107 was elected leader, sending a full tablet report...
I20260807 11:26:21.693563 31413 consensus_queue.cc:260] T 210aebcd7db5479c9dd5c4bd4c7e431c P 512cff87a93e4ad1872d512d1ed7b9e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66b68334ea547f19fe5ffc897229374" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 44821 } } peers { permanent_uuid: "512cff87a93e4ad1872d512d1ed7b9e7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 38697 } } peers { permanent_uuid: "d34fc44d7aa0493bae4e72c5c23948ef" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38887 } }
I20260807 11:26:21.694164 31410 raft_consensus.cc:348] T 210aebcd7db5479c9dd5c4bd4c7e431c P d34fc44d7aa0493bae4e72c5c23948ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66b68334ea547f19fe5ffc897229374" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 44821 } } peers { permanent_uuid: "512cff87a93e4ad1872d512d1ed7b9e7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 38697 } } peers { permanent_uuid: "d34fc44d7aa0493bae4e72c5c23948ef" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38887 } }
I20260807 11:26:21.694710 31410 raft_consensus.cc:374] T 210aebcd7db5479c9dd5c4bd4c7e431c P d34fc44d7aa0493bae4e72c5c23948ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:26:21.694896 31410 raft_consensus.cc:729] T 210aebcd7db5479c9dd5c4bd4c7e431c P d34fc44d7aa0493bae4e72c5c23948ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d34fc44d7aa0493bae4e72c5c23948ef, State: Initialized, Role: FOLLOWER
I20260807 11:26:21.695672 31410 consensus_queue.cc:260] T 210aebcd7db5479c9dd5c4bd4c7e431c P d34fc44d7aa0493bae4e72c5c23948ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66b68334ea547f19fe5ffc897229374" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 44821 } } peers { permanent_uuid: "512cff87a93e4ad1872d512d1ed7b9e7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 38697 } } peers { permanent_uuid: "d34fc44d7aa0493bae4e72c5c23948ef" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38887 } }
I20260807 11:26:21.702325 31413 ts_tablet_manager.cc:1428] T 210aebcd7db5479c9dd5c4bd4c7e431c P 512cff87a93e4ad1872d512d1ed7b9e7: Time spent starting tablet: real 0.021s user 0.019s sys 0.001s
I20260807 11:26:21.702384 31410 ts_tablet_manager.cc:1428] T 210aebcd7db5479c9dd5c4bd4c7e431c P d34fc44d7aa0493bae4e72c5c23948ef: Time spent starting tablet: real 0.022s user 0.016s sys 0.004s
I20260807 11:26:21.702713 31384 heartbeater.cc:503] Master 127.16.197.62:33107 was elected leader, sending a full tablet report...
I20260807 11:26:21.744179 31419 raft_consensus.cc:482] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:26:21.744530 31419 raft_consensus.cc:504] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66b68334ea547f19fe5ffc897229374" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 44821 } } peers { permanent_uuid: "512cff87a93e4ad1872d512d1ed7b9e7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 38697 } } peers { permanent_uuid: "d34fc44d7aa0493bae4e72c5c23948ef" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38887 } }
I20260807 11:26:21.746598 31419 leader_election.cc:288] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 512cff87a93e4ad1872d512d1ed7b9e7 (127.16.197.3:38697), d34fc44d7aa0493bae4e72c5c23948ef (127.16.197.2:38887)
I20260807 11:26:21.756377 31403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:21.747911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:33693 (local address 127.16.197.2:38887)
0807 11:26:21.748122 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:21.748128 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:21.748296 (+ 168us) server_negotiation.cc:407] Connection header received
0807 11:26:21.748670 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:21.748675 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:21.748755 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:21.748853 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:26:21.749569 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.750801 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.754211 (+ 3410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.755202 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.755428 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:21.755515 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:21.755981 (+ 466us) server_negotiation.cc:299] Negotiation successful
0807 11:26:21.756122 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 11:26:21.756551 31422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:21.746912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:38887 (local address 127.16.197.1:33693)
0807 11:26:21.748132 (+ 1220us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:21.748167 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:21.748451 (+ 284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:21.748887 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:21.748891 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:21.748915 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:26:21.749398 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:21.749411 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.751057 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.751063 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:21.753984 (+ 2921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:21.753995 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.755451 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.755458 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:21.755681 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:21.755746 (+ 65us) client_negotiation.cc:769] Sending connection context
0807 11:26:21.755925 (+ 179us) client_negotiation.cc:240] Negotiation successful
0807 11:26:21.756088 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1047,"mutex_wait_us":371,"thread_start_us":583,"threads_started":1}
I20260807 11:26:21.756662 31404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:21.747234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:50729 (local address 127.16.197.3:38697)
0807 11:26:21.747471 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:21.747480 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:21.748604 (+ 1124us) server_negotiation.cc:407] Connection header received
0807 11:26:21.748870 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:21.748876 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:21.748964 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:21.749077 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:26:21.749810 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.751043 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.754165 (+ 3122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.755424 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.755761 (+ 337us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:21.755833 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:21.756207 (+ 374us) server_negotiation.cc:299] Negotiation successful
0807 11:26:21.756384 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 11:26:21.757467 31423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:21.747074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:38697 (local address 127.16.197.1:50729)
0807 11:26:21.748494 (+ 1420us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:21.748514 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:21.748705 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:21.749116 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:21.749124 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:21.749152 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:26:21.749612 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:21.749625 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.751249 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.751259 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:21.753959 (+ 2700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:21.753983 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.755808 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.755818 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:21.756007 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:21.756040 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 11:26:21.756865 (+ 825us) client_negotiation.cc:240] Negotiation successful
0807 11:26:21.757032 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1288,"mutex_wait_us":29,"thread_start_us":636,"threads_started":1}
I20260807 11:26:21.757984 31192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "210aebcd7db5479c9dd5c4bd4c7e431c" candidate_uuid: "b66b68334ea547f19fe5ffc897229374" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d34fc44d7aa0493bae4e72c5c23948ef" is_pre_election: true
I20260807 11:26:21.758605 31192 raft_consensus.cc:2454] T 210aebcd7db5479c9dd5c4bd4c7e431c P d34fc44d7aa0493bae4e72c5c23948ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b66b68334ea547f19fe5ffc897229374 in term 0.
I20260807 11:26:21.758594 31336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "210aebcd7db5479c9dd5c4bd4c7e431c" candidate_uuid: "b66b68334ea547f19fe5ffc897229374" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "512cff87a93e4ad1872d512d1ed7b9e7" is_pre_election: true
I20260807 11:26:21.759220 31336 raft_consensus.cc:2454] T 210aebcd7db5479c9dd5c4bd4c7e431c P 512cff87a93e4ad1872d512d1ed7b9e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b66b68334ea547f19fe5ffc897229374 in term 0.
I20260807 11:26:21.759379 30969 leader_election.cc:302] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b66b68334ea547f19fe5ffc897229374, d34fc44d7aa0493bae4e72c5c23948ef; no voters:
I20260807 11:26:21.759852 31419 raft_consensus.cc:2781] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:26:21.760001 31419 raft_consensus.cc:482] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:26:21.760104 31419 raft_consensus.cc:3037] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:26:21.763222 31419 raft_consensus.cc:504] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66b68334ea547f19fe5ffc897229374" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 44821 } } peers { permanent_uuid: "512cff87a93e4ad1872d512d1ed7b9e7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 38697 } } peers { permanent_uuid: "d34fc44d7aa0493bae4e72c5c23948ef" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38887 } }
I20260807 11:26:21.764106 31419 leader_election.cc:288] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374 [CANDIDATE]: Term 1 election: Requested vote from peers 512cff87a93e4ad1872d512d1ed7b9e7 (127.16.197.3:38697), d34fc44d7aa0493bae4e72c5c23948ef (127.16.197.2:38887)
I20260807 11:26:21.764462 31336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "210aebcd7db5479c9dd5c4bd4c7e431c" candidate_uuid: "b66b68334ea547f19fe5ffc897229374" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "512cff87a93e4ad1872d512d1ed7b9e7"
I20260807 11:26:21.764530 31192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "210aebcd7db5479c9dd5c4bd4c7e431c" candidate_uuid: "b66b68334ea547f19fe5ffc897229374" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d34fc44d7aa0493bae4e72c5c23948ef"
I20260807 11:26:21.764709 31336 raft_consensus.cc:3037] T 210aebcd7db5479c9dd5c4bd4c7e431c P 512cff87a93e4ad1872d512d1ed7b9e7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:26:21.764729 31192 raft_consensus.cc:3037] T 210aebcd7db5479c9dd5c4bd4c7e431c P d34fc44d7aa0493bae4e72c5c23948ef [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:26:21.768132 31336 raft_consensus.cc:2454] T 210aebcd7db5479c9dd5c4bd4c7e431c P 512cff87a93e4ad1872d512d1ed7b9e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b66b68334ea547f19fe5ffc897229374 in term 1.
I20260807 11:26:21.768155 31192 raft_consensus.cc:2454] T 210aebcd7db5479c9dd5c4bd4c7e431c P d34fc44d7aa0493bae4e72c5c23948ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b66b68334ea547f19fe5ffc897229374 in term 1.
I20260807 11:26:21.768605 30968 leader_election.cc:302] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 512cff87a93e4ad1872d512d1ed7b9e7, b66b68334ea547f19fe5ffc897229374; no voters:
I20260807 11:26:21.768956 31419 raft_consensus.cc:2781] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:26:21.769512 31419 raft_consensus.cc:686] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374 [term 1 LEADER]: Becoming Leader. State: Replica: b66b68334ea547f19fe5ffc897229374, State: Running, Role: LEADER
I20260807 11:26:21.770103 31419 consensus_queue.cc:237] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66b68334ea547f19fe5ffc897229374" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 44821 } } peers { permanent_uuid: "512cff87a93e4ad1872d512d1ed7b9e7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 38697 } } peers { permanent_uuid: "d34fc44d7aa0493bae4e72c5c23948ef" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38887 } }
I20260807 11:26:21.775905 30879 catalog_manager.cc:5495] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374 reported cstate change: term changed from 0 to 1, leader changed from <none> to b66b68334ea547f19fe5ffc897229374 (127.16.197.1). New cstate: current_term: 1 leader_uuid: "b66b68334ea547f19fe5ffc897229374" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66b68334ea547f19fe5ffc897229374" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 44821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "512cff87a93e4ad1872d512d1ed7b9e7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 38697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d34fc44d7aa0493bae4e72c5c23948ef" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38887 } health_report { overall_health: UNKNOWN } } }
I20260807 11:26:21.812702 31402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:21.806509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55722 (local address 127.16.197.1:44821)
0807 11:26:21.806724 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:21.806729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:21.806748 (+ 19us) server_negotiation.cc:407] Connection header received
0807 11:26:21.806846 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:21.806849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:21.806927 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:21.807122 (+ 195us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:26:21.807561 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.808338 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.810351 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.810764 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.810892 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:21.811505 (+ 613us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:26:21.812054 (+ 549us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:26:21.812170 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:21.812327 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 11:26:21.812400 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 11:26:21.822098 31336 raft_consensus.cc:1261] T 210aebcd7db5479c9dd5c4bd4c7e431c P 512cff87a93e4ad1872d512d1ed7b9e7 [term 1 FOLLOWER]: Refusing update from remote peer b66b68334ea547f19fe5ffc897229374: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:26:21.822095 31192 raft_consensus.cc:1261] T 210aebcd7db5479c9dd5c4bd4c7e431c P d34fc44d7aa0493bae4e72c5c23948ef [term 1 FOLLOWER]: Refusing update from remote peer b66b68334ea547f19fe5ffc897229374: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:26:21.823242 31424 consensus_queue.cc:1035] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374 [LEADER]: Connected to new peer: Peer: permanent_uuid: "512cff87a93e4ad1872d512d1ed7b9e7" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 38697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:26:21.823662 31419 consensus_queue.cc:1035] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d34fc44d7aa0493bae4e72c5c23948ef" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 11:26:21.823949 31385 tablet.cc:2369] T 210aebcd7db5479c9dd5c4bd4c7e431c P 512cff87a93e4ad1872d512d1ed7b9e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:26:21.840386 31427 mvcc.cc:205] Tried to move back new op lower bound from 7315873717531922432 to 7315873717335179264. Current Snapshot: MvccSnapshot[applied={T|T < 7315873717531922432}]
I20260807 11:26:21.874250 30879 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:26:21.874562 30879 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:26:21.879215 30879 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 99c363da42cd45aeb31e929a95793c36: Sending DeleteTablet for 3 replicas of tablet 210aebcd7db5479c9dd5c4bd4c7e431c
I20260807 11:26:21.880486 31018 tablet_service.cc:1514] Processing DeleteTablet for tablet 210aebcd7db5479c9dd5c4bd4c7e431c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:26:21 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:55708
I20260807 11:26:21.882009 31316 tablet_service.cc:1514] Processing DeleteTablet for tablet 210aebcd7db5479c9dd5c4bd4c7e431c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:26:21 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:49396
I20260807 11:26:21.883141 31438 tablet_replica.cc:330] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374: stopping tablet replica
I20260807 11:26:21.883611 31438 raft_consensus.cc:2229] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374 [term 1 LEADER]: Raft consensus shutting down.
I20260807 11:26:21.884378 31438 raft_consensus.cc:2258] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:26:21.884905 31171 tablet_service.cc:1514] Processing DeleteTablet for tablet 210aebcd7db5479c9dd5c4bd4c7e431c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:26:21 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:41696
I20260807 11:26:21.885040 31439 tablet_replica.cc:330] T 210aebcd7db5479c9dd5c4bd4c7e431c P 512cff87a93e4ad1872d512d1ed7b9e7: stopping tablet replica
I20260807 11:26:21.885458 31439 raft_consensus.cc:2229] T 210aebcd7db5479c9dd5c4bd4c7e431c P 512cff87a93e4ad1872d512d1ed7b9e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:26:21.885957 31439 raft_consensus.cc:2258] T 210aebcd7db5479c9dd5c4bd4c7e431c P 512cff87a93e4ad1872d512d1ed7b9e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:26:21.886766 31440 tablet_replica.cc:330] T 210aebcd7db5479c9dd5c4bd4c7e431c P d34fc44d7aa0493bae4e72c5c23948ef: stopping tablet replica
I20260807 11:26:21.887181 31440 raft_consensus.cc:2229] T 210aebcd7db5479c9dd5c4bd4c7e431c P d34fc44d7aa0493bae4e72c5c23948ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:26:21.887666 31440 raft_consensus.cc:2258] T 210aebcd7db5479c9dd5c4bd4c7e431c P d34fc44d7aa0493bae4e72c5c23948ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:26:21.889357 31438 ts_tablet_manager.cc:1905] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:26:21.889870 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 30920
I20260807 11:26:21.901062 31440 ts_tablet_manager.cc:1905] T 210aebcd7db5479c9dd5c4bd4c7e431c P d34fc44d7aa0493bae4e72c5c23948ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:26:21.902163 31439 ts_tablet_manager.cc:1905] T 210aebcd7db5479c9dd5c4bd4c7e431c P 512cff87a93e4ad1872d512d1ed7b9e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:26:21.908857 31438 ts_tablet_manager.cc:1918] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:26:21.909075 31438 log.cc:1196] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-0/wal/wals/210aebcd7db5479c9dd5c4bd4c7e431c
I20260807 11:26:21.909663 31438 ts_tablet_manager.cc:1939] T 210aebcd7db5479c9dd5c4bd4c7e431c P b66b68334ea547f19fe5ffc897229374: Deleting consensus metadata
I20260807 11:26:21.910120 31439 ts_tablet_manager.cc:1918] T 210aebcd7db5479c9dd5c4bd4c7e431c P 512cff87a93e4ad1872d512d1ed7b9e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:26:21.910291 31439 log.cc:1196] T 210aebcd7db5479c9dd5c4bd4c7e431c P 512cff87a93e4ad1872d512d1ed7b9e7: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-2/wal/wals/210aebcd7db5479c9dd5c4bd4c7e431c
I20260807 11:26:21.910799 31439 ts_tablet_manager.cc:1939] T 210aebcd7db5479c9dd5c4bd4c7e431c P 512cff87a93e4ad1872d512d1ed7b9e7: Deleting consensus metadata
I20260807 11:26:21.912679 30863 catalog_manager.cc:4841] TS 512cff87a93e4ad1872d512d1ed7b9e7 (127.16.197.3:38697): tablet 210aebcd7db5479c9dd5c4bd4c7e431c (table test-table [id=78c65b66b1464e7cb948459a6f1eefde]) successfully deleted
I20260807 11:26:21.915306 30863 catalog_manager.cc:4841] TS b66b68334ea547f19fe5ffc897229374 (127.16.197.1:44821): tablet 210aebcd7db5479c9dd5c4bd4c7e431c (table test-table [id=78c65b66b1464e7cb948459a6f1eefde]) successfully deleted
I20260807 11:26:21.915995 31440 ts_tablet_manager.cc:1918] T 210aebcd7db5479c9dd5c4bd4c7e431c P d34fc44d7aa0493bae4e72c5c23948ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:26:21.916180 31440 log.cc:1196] T 210aebcd7db5479c9dd5c4bd4c7e431c P d34fc44d7aa0493bae4e72c5c23948ef: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786101765888975-17172-0/minicluster-data/ts-1/wal/wals/210aebcd7db5479c9dd5c4bd4c7e431c
I20260807 11:26:21.917876 31440 ts_tablet_manager.cc:1939] T 210aebcd7db5479c9dd5c4bd4c7e431c P d34fc44d7aa0493bae4e72c5c23948ef: Deleting consensus metadata
I20260807 11:26:21.919792 30864 catalog_manager.cc:4841] TS d34fc44d7aa0493bae4e72c5c23948ef (127.16.197.2:38887): tablet 210aebcd7db5479c9dd5c4bd4c7e431c (table test-table [id=78c65b66b1464e7cb948459a6f1eefde]) successfully deleted
I20260807 11:26:21.928730 31402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:21.891992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55724 (local address 127.16.197.1:44821)
0807 11:26:21.892165 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:21.892171 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:21.907557 (+ 15386us) server_negotiation.cc:407] Connection header received
0807 11:26:21.907767 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:21.907773 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:21.907943 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:21.908068 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:21.913177 (+ 5109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.914471 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.916418 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:21.917118 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:21.917252 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:21.920869 (+ 3617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:21.920885 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:21.920905 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:21.920953 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:21.925244 (+ 4291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:21.926210 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:21.926219 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:21.926224 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:21.926413 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:21.926870 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:21.926876 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:21.926879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:21.927367 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:21.927684 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:21.928105 (+ 421us) server_negotiation.cc:299] Negotiation successful
0807 11:26:21.928349 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 11:26:21.929494 31082 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:26:22.036231 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 30920
I20260807 11:26:22.056258 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 31091
I20260807 11:26:22.070796 31403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:22.057178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41706 (local address 127.16.197.2:38887)
0807 11:26:22.057355 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:22.057360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:22.057379 (+ 19us) server_negotiation.cc:407] Connection header received
0807 11:26:22.057526 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:22.057530 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:22.057604 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:22.057690 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:22.059161 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:22.059921 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:22.061286 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:22.061690 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:22.061784 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:22.064775 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:22.064791 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:22.064807 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:22.064842 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:22.067995 (+ 3153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:22.068789 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:22.068796 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:22.068799 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:22.068973 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:22.069313 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:22.069316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:22.069318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:22.069671 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:22.069919 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:22.070194 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 11:26:22.070361 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 11:26:22.071563 31237 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:26:22.164143 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 31091
I20260807 11:26:22.184610 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 31246
I20260807 11:26:22.198139 31404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:22.185537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49406 (local address 127.16.197.3:38697)
0807 11:26:22.185707 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:22.185712 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:22.185731 (+ 19us) server_negotiation.cc:407] Connection header received
0807 11:26:22.185933 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:22.185937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:22.186008 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:22.186105 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:22.187650 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:22.188387 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:22.189614 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:22.189990 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:22.190086 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:22.192796 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:22.192810 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:22.192826 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:22.192858 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:22.195669 (+ 2811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:22.196317 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:22.196325 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:22.196327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:22.196456 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:22.196771 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:22.196774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:22.196776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:22.197127 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:22.197345 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:22.197607 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 11:26:22.197771 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 11:26:22.198827 31378 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:26:22.289160 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 31246
I20260807 11:26:22.308427 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 30843
I20260807 11:26:22.309754 30912 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:26:22.426503 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 30843
I20260807 11:26:22.454486 17172 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 11:26:22.800597 17172 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 11:26:22.800860 17172 mini_ranger.cc:66] Stopping Ranger...
I20260807 11:26:23.146788 17172 mini_ranger.cc:68] Stopped Ranger
2026-08-07 11:26:23.147 UTC [29836] LOG: received smart shutdown request
2026-08-07 11:26:23.150 UTC [29836] LOG: background worker "logical replication launcher" (PID 29845) exited with exit code 1
2026-08-07 11:26:23.150 UTC [29840] LOG: shutting down
2026-08-07 11:26:23.338 UTC [29836] LOG: database system is shut down
2026-08-07T11:26:23Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (54032 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:26:23 dist-test-slave-2w3w krb5kdc[31455](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:26:23 dist-test-slave-2w3w krb5kdc[31455](info): set up 2 sockets
Aug 07 11:26:23 dist-test-slave-2w3w krb5kdc[31455](info): commencing operation
krb5kdc: starting...
W20260807 11:26:24.626886 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.004s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:26:24 dist-test-slave-2w3w krb5kdc[31455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:26:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:26:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:26:24.899161 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:45723
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:45625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:45723,127.16.197.61:33985,127.16.197.60:34373
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:26:25.252703 31468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:26:25.253024 31468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:26:25.253190 31468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:26:25.260581 31468 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:26:25.260695 31468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:26:25.260797 31468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:26:25.260895 31468 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:26:25.260937 31468 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:26:25.271319 31468 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:45625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:45723,127.16.197.61:33985,127.16.197.60:34373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:45723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:26:25.272742 31468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:26:25.274134 31468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:26:25.286543 31474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:25.286553 31475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:25.286595 31477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:25.287889 31468 server_base.cc:1034] running on GCE node
I20260807 11:26:25.288714 31468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:26:25.290263 31468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:26:25.291517 31468 hybrid_clock.cc:648] HybridClock initialized: now 1786101985291461 us; error 36 us; skew 500 ppm
I20260807 11:26:25.291960 31468 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:26:25 dist-test-slave-2w3w krb5kdc[31455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101985, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:26:25.295256 31468 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:26:25.296928 31468 webserver.cc:460] Webserver started at http://127.16.197.62:45061/ using document root <none> and password file <none>
I20260807 11:26:25.297660 31468 fs_manager.cc:359] Metadata directory not provided
I20260807 11:26:25.297813 31468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:26:25.298151 31468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:26:25.301689 31468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "d86a6a8cf64f4752af989c53294584f7"
format_stamp: "Formatted at 2026-08-07 11:26:25 on dist-test-slave-2w3w"
server_key: "8e428521dc51a2f54d0aa2d55d70a0cc"
server_key_iv: "83827e05e42d1f456052599eae502a74"
server_key_version: "encryptionkey@0"
I20260807 11:26:25.302634 31468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "d86a6a8cf64f4752af989c53294584f7"
format_stamp: "Formatted at 2026-08-07 11:26:25 on dist-test-slave-2w3w"
server_key: "8e428521dc51a2f54d0aa2d55d70a0cc"
server_key_iv: "83827e05e42d1f456052599eae502a74"
server_key_version: "encryptionkey@0"
I20260807 11:26:25.308266 31468 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 11:26:25.312067 31485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:25.313411 31468 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 11:26:25.313613 31468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "d86a6a8cf64f4752af989c53294584f7"
format_stamp: "Formatted at 2026-08-07 11:26:25 on dist-test-slave-2w3w"
server_key: "8e428521dc51a2f54d0aa2d55d70a0cc"
server_key_iv: "83827e05e42d1f456052599eae502a74"
server_key_version: "encryptionkey@0"
I20260807 11:26:25.313836 31468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:26:25.333778 31468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:26:25.337975 31468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:26:25.338296 31468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:26:25.358141 31468 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:45723
I20260807 11:26:25.358173 31539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:45723 every 8 connection(s)
I20260807 11:26:25.359637 31468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:26:25.363451 31540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:26:25.366667 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 31468
I20260807 11:26:25.366940 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:26:25.367823 17172 external_mini_cluster.cc:1428] Setting key a468af0bf67b88df672088ff775a8ae6
I20260807 11:26:25.372684 31540 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45723 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 11:26:25 dist-test-slave-2w3w krb5kdc[31455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101985, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.197.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.61@KRBTEST.COM" created.
I20260807 11:26:25.400110 31543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:25.376950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47286 (local address 127.16.197.62:45723)
0807 11:26:25.379145 (+ 2195us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:25.379151 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:25.379194 (+ 43us) server_negotiation.cc:407] Connection header received
0807 11:26:25.380999 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:25.381010 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:25.381306 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:25.381536 (+ 230us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:25.382718 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:25.384797 (+ 2079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:25.386239 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:25.387283 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:25.387504 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:25.391133 (+ 3629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:25.391146 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:25.391166 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:25.391220 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:25.395369 (+ 4149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:25.396156 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:25.396163 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:25.396166 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:25.396333 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:25.396820 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:25.396823 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:25.396825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:25.397410 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:25.397572 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:25.398287 (+ 715us) server_negotiation.cc:299] Negotiation successful
0807 11:26:25.398798 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":1957,"spinlock_wait_cycles":32640,"thread_start_us":1717,"threads_started":1}
I20260807 11:26:25.400149 31542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:25.376950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45723 (local address 127.0.0.1:47286)
0807 11:26:25.378088 (+ 1138us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:25.378159 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:25.380585 (+ 2426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:25.381561 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:25.381566 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:25.382148 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:26:25.382508 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:25.382526 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:25.384973 (+ 2447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:25.384977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:25.386079 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:25.386091 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:25.387543 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:25.387556 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:25.387684 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:25.388394 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:26:25.388417 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:26:25.390833 (+ 2416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:26:25.395547 (+ 4714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:25.395563 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:25.395578 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:25.396020 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:25.396449 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:25.396452 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:25.396454 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:25.396703 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:25.397549 (+ 846us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:26:25.397555 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:26:25.397896 (+ 341us) client_negotiation.cc:769] Sending connection context
0807 11:26:25.398148 (+ 252us) client_negotiation.cc:240] Negotiation successful
0807 11:26:25.398992 (+ 844us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"mutex_wait_us":245,"thread_start_us":556,"threads_started":1}
I20260807 11:26:25.403258 31540 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.61" port: 33985 } has no permanent_uuid. Determining permanent_uuid...
I20260807 11:26:25.404402 31542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:25.403983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:33985 (local address 127.0.0.1:59442)
0807 11:26:25.404160 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:25.404279 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:33985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260807 11:26:25.404750 31487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.197.61:33985: connect: Connection refused (error 111)
I20260807 11:26:25.406378 31542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:25.406021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:33985 (local address 127.0.0.1:59448)
0807 11:26:25.406159 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:25.406227 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:33985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 11:26:25.406669 31540 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.61:33985: Network error: Client connection negotiation failed: client connection to 127.16.197.61:33985: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.61.keytab.
Entry for principal kudu/127.16.197.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.61.keytab.
I20260807 11:26:25.432559 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.61:33985
--webserver_interface=127.16.197.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.61.keytab
--principal=kudu/127.16.197.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:45625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:45723,127.16.197.61:33985,127.16.197.60:34373
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 11:26:25.456099 31540 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.61" port: 33985 } attempt: 1
I20260807 11:26:25.457505 31542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:25.456838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:33985 (local address 127.0.0.1:59462)
0807 11:26:25.457123 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:25.457305 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:33985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 11:26:25.458655 31542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:25.458226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:33985 (local address 127.0.0.1:59472)
0807 11:26:25.458386 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:25.458477 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:33985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 11:26:25.458859 31540 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.61:33985: Network error: Client connection negotiation failed: client connection to 127.16.197.61:33985: connect: Connection refused (error 111)
I20260807 11:26:25.527140 31540 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.61" port: 33985 } attempt: 2
I20260807 11:26:25.528517 31542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:25.527939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:33985 (local address 127.0.0.1:59484)
0807 11:26:25.528220 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:25.528335 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:33985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 11:26:25.529618 31542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:25.529189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:33985 (local address 127.0.0.1:59486)
0807 11:26:25.529358 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:25.529445 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:33985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 11:26:25.529812 31540 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.61:33985: Network error: Client connection negotiation failed: client connection to 127.16.197.61:33985: connect: Connection refused (error 111)
I20260807 11:26:25.621088 31540 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.61" port: 33985 } attempt: 3
I20260807 11:26:25.622439 31542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:25.621876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:33985 (local address 127.0.0.1:59490)
0807 11:26:25.622113 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:25.622262 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:33985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 11:26:25.623703 31542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:25.623224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:33985 (local address 127.0.0.1:59496)
0807 11:26:25.623401 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:25.623494 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:33985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 11:26:25.623981 31540 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.61:33985: Network error: Client connection negotiation failed: client connection to 127.16.197.61:33985: connect: Connection refused (error 111)
I20260807 11:26:25.767257 31540 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.61" port: 33985 } attempt: 4
I20260807 11:26:25.768585 31542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:25.768009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:33985 (local address 127.0.0.1:59508)
0807 11:26:25.768362 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:25.768449 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:33985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187}
I20260807 11:26:25.769655 31542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:25.769294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:33985 (local address 127.0.0.1:59512)
0807 11:26:25.769437 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:25.769532 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:33985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 11:26:25.769941 31540 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.61:33985: Network error: Client connection negotiation failed: client connection to 127.16.197.61:33985: connect: Connection refused (error 111)
W20260807 11:26:25.788105 31547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:26:25.788468 31547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:26:25.788630 31547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:26:25.796394 31547 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:26:25.796555 31547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:26:25.796672 31547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:26:25.796787 31547 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:26:25.796869 31547 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:26:25.807112 31547 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:45625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:45723,127.16.197.61:33985,127.16.197.60:34373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.61.keytab
--principal=kudu/127.16.197.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.61:33985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.16.197.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:26:25.808573 31547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:26:25.809944 31547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:26:25.822268 31552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:25.822233 31553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:25.822531 31555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:25.823170 31547 server_base.cc:1034] running on GCE node
I20260807 11:26:25.824122 31547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:26:25.825574 31547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:26:25.826803 31547 hybrid_clock.cc:648] HybridClock initialized: now 1786101985826760 us; error 40 us; skew 500 ppm
I20260807 11:26:25.827221 31547 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:26:25 dist-test-slave-2w3w krb5kdc[31455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101985, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:26:25.833446 31547 init.cc:377] Logged in from keytab as kudu/127.16.197.61@KRBTEST.COM (short username kudu)
I20260807 11:26:25.835134 31547 webserver.cc:460] Webserver started at http://127.16.197.61:36661/ using document root <none> and password file <none>
I20260807 11:26:25.835897 31547 fs_manager.cc:359] Metadata directory not provided
I20260807 11:26:25.836061 31547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:26:25.836383 31547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:26:25.839639 31547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-1/data/instance:
uuid: "d7c9f1e595fd472e9c15963cdad9d00a"
format_stamp: "Formatted at 2026-08-07 11:26:25 on dist-test-slave-2w3w"
server_key: "1c5f4d061ce33fa21f00859e9adf8fcc"
server_key_iv: "980cca96035c9d042b443eaacf2860b3"
server_key_version: "encryptionkey@0"
I20260807 11:26:25.840574 31547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-1/wal/instance:
uuid: "d7c9f1e595fd472e9c15963cdad9d00a"
format_stamp: "Formatted at 2026-08-07 11:26:25 on dist-test-slave-2w3w"
server_key: "1c5f4d061ce33fa21f00859e9adf8fcc"
server_key_iv: "980cca96035c9d042b443eaacf2860b3"
server_key_version: "encryptionkey@0"
I20260807 11:26:25.846071 31547 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 11:26:25.849598 31562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:25.850937 31547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 11:26:25.851186 31547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-1/wal
uuid: "d7c9f1e595fd472e9c15963cdad9d00a"
format_stamp: "Formatted at 2026-08-07 11:26:25 on dist-test-slave-2w3w"
server_key: "1c5f4d061ce33fa21f00859e9adf8fcc"
server_key_iv: "980cca96035c9d042b443eaacf2860b3"
server_key_version: "encryptionkey@0"
I20260807 11:26:25.851387 31547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:26:25.871809 31547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:26:25.876160 31547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:26:25.876470 31547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:26:25.896656 31547 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.61:33985
I20260807 11:26:25.896690 31615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.61:33985 every 8 connection(s)
I20260807 11:26:25.898121 31547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-1/data/info.pb
I20260807 11:26:25.899341 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 31547
I20260807 11:26:25.899576 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-1/wal/instance
I20260807 11:26:25.900045 17172 external_mini_cluster.cc:1428] Setting key 3675672c36c91588352aafb4b0f5a5e6
I20260807 11:26:25.902297 31616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:26:25.909678 31616 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45723 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.16.197.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.60@KRBTEST.COM" created.
Aug 07 11:26:25 dist-test-slave-2w3w krb5kdc[31455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101985, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.61@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:26:25.937778 31619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:25.913382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47288 (local address 127.16.197.62:45723)
0807 11:26:25.913957 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:25.913964 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:25.915081 (+ 1117us) server_negotiation.cc:407] Connection header received
0807 11:26:25.918199 (+ 3118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:25.918204 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:25.918279 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:25.918387 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:25.920694 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:25.921422 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:25.924199 (+ 2777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:25.924631 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:25.924767 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:25.931217 (+ 6450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:25.931223 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:25.931226 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:25.931275 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:25.934136 (+ 2861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:25.935272 (+ 1136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:25.935277 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:25.935279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:25.935406 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:25.935915 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:25.935919 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:25.935920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:25.936212 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:25.936326 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:25.937322 (+ 996us) server_negotiation.cc:299] Negotiation successful
0807 11:26:25.937511 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":290,"threads_started":1}
I20260807 11:26:25.939260 31620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:25.913953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45723 (local address 127.0.0.1:47288)
0807 11:26:25.914867 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:25.914918 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:25.917888 (+ 2970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:25.918692 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:25.918705 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:25.919567 (+ 862us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:26:25.920479 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:25.920508 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:25.921619 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:25.921627 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:25.924029 (+ 2402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:25.924050 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:25.924912 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:25.924921 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:25.925257 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:25.926030 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:26:25.926068 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:26:25.930891 (+ 4823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:26:25.934316 (+ 3425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:25.934328 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:25.934350 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:25.935140 (+ 790us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:25.935521 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:25.935528 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:25.935534 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:25.935786 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:25.936349 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:26:25.936360 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:26:25.936880 (+ 520us) client_negotiation.cc:769] Sending connection context
0807 11:26:25.937211 (+ 331us) client_negotiation.cc:240] Negotiation successful
0807 11:26:25.937796 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":329,"threads_started":1}
I20260807 11:26:25.940958 31616 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.61" port: 33985 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 11:26:25 dist-test-slave-2w3w krb5kdc[31455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101985, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.61@KRBTEST.COM for kudu/127.16.197.61@KRBTEST.COM
I20260807 11:26:25.956291 31620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:25.941851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:33985 (local address 127.0.0.1:59514)
0807 11:26:25.942052 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:25.942078 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:25.942216 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:25.943283 (+ 1067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:25.943287 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:25.943750 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:26:25.943897 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:25.943905 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:25.945260 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:25.945268 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:25.946109 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:25.946119 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:25.946986 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:25.946993 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:25.947203 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:25.947809 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:26:25.947827 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:26:25.950123 (+ 2296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:26:25.953498 (+ 3375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:25.953505 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:25.953509 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:25.953898 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:25.954287 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:25.954292 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:25.954295 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:25.954452 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:25.955204 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:26:25.955212 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:26:25.955363 (+ 151us) client_negotiation.cc:769] Sending connection context
0807 11:26:25.955519 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 11:26:25.955812 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 11:26:25.956362 31622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:25.941875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59514 (local address 127.16.197.61:33985)
0807 11:26:25.942830 (+ 955us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:25.942836 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:25.942862 (+ 26us) server_negotiation.cc:407] Connection header received
0807 11:26:25.942903 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:25.942910 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:25.943139 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:25.943297 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:25.944059 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:25.945127 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:25.946245 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:25.946789 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:25.946924 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:25.950353 (+ 3429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:25.950365 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:25.950380 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:25.950409 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:25.953358 (+ 2949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:25.954045 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:25.954052 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:25.954054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:25.954165 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:25.954576 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:25.954585 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:25.954587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:25.955079 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:25.955312 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:25.955656 (+ 344us) server_negotiation.cc:299] Negotiation successful
0807 11:26:25.955926 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":798,"thread_start_us":637,"threads_started":1}
I20260807 11:26:25.959000 31616 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } has no permanent_uuid. Determining permanent_uuid...
I20260807 11:26:25.960299 31620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:25.959795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:34373 (local address 127.0.0.1:52672)
0807 11:26:25.960005 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:25.960140 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:34373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 11:26:25.960633 31565 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.197.60:34373: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.60.keytab.
Entry for principal kudu/127.16.197.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.60.keytab.
I20260807 11:26:25.962517 31620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:25.962137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:34373 (local address 127.0.0.1:52676)
0807 11:26:25.962307 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:25.962388 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:34373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 11:26:25.962689 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.60:34373
--webserver_interface=127.16.197.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.60.keytab
--principal=kudu/127.16.197.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:45625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:45723,127.16.197.61:33985,127.16.197.60:34373
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:26:25.962772 31616 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.60:34373: Network error: Client connection negotiation failed: client connection to 127.16.197.60:34373: connect: Connection refused (error 111)
I20260807 11:26:26.012113 31616 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } attempt: 1
I20260807 11:26:26.013514 31620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.012828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:34373 (local address 127.0.0.1:52690)
0807 11:26:26.013117 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:26.013277 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:34373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260807 11:26:26.015027 31620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.014438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:34373 (local address 127.0.0.1:52702)
0807 11:26:26.014672 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:26.014803 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:34373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260807 11:26:26.015309 31616 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.60:34373: Network error: Client connection negotiation failed: client connection to 127.16.197.60:34373: connect: Connection refused (error 111)
I20260807 11:26:26.069208 31540 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.61" port: 33985 } attempt: 5
Aug 07 11:26:26 dist-test-slave-2w3w krb5kdc[31455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101985, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for kudu/127.16.197.61@KRBTEST.COM
I20260807 11:26:26.083563 31616 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } attempt: 2
I20260807 11:26:26.084775 31620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.084226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:34373 (local address 127.0.0.1:52716)
0807 11:26:26.084445 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:26.084572 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:34373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260807 11:26:26.085119 31622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.070183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59520 (local address 127.16.197.61:33985)
0807 11:26:26.070411 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:26.070418 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:26.070443 (+ 25us) server_negotiation.cc:407] Connection header received
0807 11:26:26.070542 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:26.070547 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:26.070628 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:26.070742 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:26.071624 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.072728 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.073892 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.074586 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.074746 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:26.078341 (+ 3595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:26.078349 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:26.078357 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:26.078409 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:26.082090 (+ 3681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:26.082870 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:26.082876 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:26.082879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:26.083075 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:26.083474 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:26.083479 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:26.083482 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:26.083896 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:26.084057 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:26.084584 (+ 527us) server_negotiation.cc:299] Negotiation successful
0807 11:26:26.084814 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 11:26:26.085134 31542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.070030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:33985 (local address 127.0.0.1:59520)
0807 11:26:26.070269 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:26.070293 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:26.070420 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:26.070778 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:26.070781 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:26.071235 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:26:26.071435 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:26.071444 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.072880 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.072884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:26.073739 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:26.073750 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.074774 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.074778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:26.074933 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:26.075660 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:26:26.075678 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:26:26.078102 (+ 2424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:26:26.082272 (+ 4170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:26.082278 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:26.082281 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:26.082730 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:26.083197 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:26.083200 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:26.083203 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:26.083353 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:26.084042 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:26:26.084050 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:26:26.084239 (+ 189us) client_negotiation.cc:769] Sending connection context
0807 11:26:26.084439 (+ 200us) client_negotiation.cc:240] Negotiation successful
0807 11:26:26.084764 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260807 11:26:26.086364 31620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.085962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:34373 (local address 127.0.0.1:52720)
0807 11:26:26.086117 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:26.086224 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:34373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 11:26:26.086421 31540 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } has no permanent_uuid. Determining permanent_uuid...
W20260807 11:26:26.087031 31616 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.60:34373: Network error: Client connection negotiation failed: client connection to 127.16.197.60:34373: connect: Connection refused (error 111)
I20260807 11:26:26.087711 31542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.087323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:34373 (local address 127.0.0.1:52722)
0807 11:26:26.087485 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:26.087583 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:34373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 11:26:26.088713 31542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.088383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:34373 (local address 127.0.0.1:52734)
0807 11:26:26.088527 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:26.088615 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:34373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 11:26:26.088929 31540 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.60:34373: Network error: Client connection negotiation failed: client connection to 127.16.197.60:34373: connect: Connection refused (error 111)
I20260807 11:26:26.140223 31540 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } attempt: 1
I20260807 11:26:26.141505 31542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.140989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:34373 (local address 127.0.0.1:52738)
0807 11:26:26.141210 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:26.141316 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:34373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 11:26:26.142596 31542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.142239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:34373 (local address 127.0.0.1:52746)
0807 11:26:26.142399 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:26.142483 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:34373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 11:26:26.142880 31540 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.60:34373: Network error: Client connection negotiation failed: client connection to 127.16.197.60:34373: connect: Connection refused (error 111)
I20260807 11:26:26.178287 31616 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } attempt: 3
I20260807 11:26:26.179541 31620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.179067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:34373 (local address 127.0.0.1:52762)
0807 11:26:26.179263 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:26.179373 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:34373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 11:26:26.180697 31620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.180244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:34373 (local address 127.0.0.1:52772)
0807 11:26:26.180488 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:26.180570 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:34373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
W20260807 11:26:26.180902 31616 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.60:34373: Network error: Client connection negotiation failed: client connection to 127.16.197.60:34373: connect: Connection refused (error 111)
I20260807 11:26:26.211153 31540 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } attempt: 2
I20260807 11:26:26.212512 31542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.211940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:34373 (local address 127.0.0.1:52788)
0807 11:26:26.212236 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:26.212331 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:34373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 11:26:26.213528 31542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.213179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:34373 (local address 127.0.0.1:52800)
0807 11:26:26.213344 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:26.213420 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:34373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 11:26:26.213754 31540 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.60:34373: Network error: Client connection negotiation failed: client connection to 127.16.197.60:34373: connect: Connection refused (error 111)
W20260807 11:26:26.316664 31624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:26:26.317028 31624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:26:26.317185 31624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 11:26:26.319953 31540 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } attempt: 3
I20260807 11:26:26.321070 31542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.320592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:34373 (local address 127.0.0.1:52806)
0807 11:26:26.320797 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:26.320904 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:34373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 11:26:26.322098 31542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.321701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:34373 (local address 127.0.0.1:52822)
0807 11:26:26.321926 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:26.322001 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:34373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
W20260807 11:26:26.322311 31540 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.60:34373: Network error: Client connection negotiation failed: client connection to 127.16.197.60:34373: connect: Connection refused (error 111)
I20260807 11:26:26.324093 31616 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } attempt: 4
I20260807 11:26:26.325140 31620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.324620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:34373 (local address 127.0.0.1:52826)
0807 11:26:26.324858 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:26.324960 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:34373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260807 11:26:26.325281 31624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:26:26.325410 31624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:26:26.325492 31624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:26:26.325592 31624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:26:26.325665 31624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:26:26.326262 31620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.325884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:34373 (local address 127.0.0.1:52832)
0807 11:26:26.326041 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:26.326127 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:34373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 11:26:26.326503 31616 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.60:34373: Network error: Client connection negotiation failed: client connection to 127.16.197.60:34373: connect: Connection refused (error 111)
I20260807 11:26:26.336259 31624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:45625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:45723,127.16.197.61:33985,127.16.197.60:34373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.60.keytab
--principal=kudu/127.16.197.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.60:34373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.16.197.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:26:26.337659 31624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:26:26.339161 31624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:26:26.351276 31632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:26.351351 31631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:26.351614 31635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:26.352401 31624 server_base.cc:1034] running on GCE node
I20260807 11:26:26.353250 31624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:26:26.354664 31624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:26:26.355942 31624 hybrid_clock.cc:648] HybridClock initialized: now 1786101986355851 us; error 74 us; skew 500 ppm
I20260807 11:26:26.356343 31624 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:26:26 dist-test-slave-2w3w krb5kdc[31455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101986, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:26:26.360076 31624 init.cc:377] Logged in from keytab as kudu/127.16.197.60@KRBTEST.COM (short username kudu)
I20260807 11:26:26.366542 31624 webserver.cc:460] Webserver started at http://127.16.197.60:44045/ using document root <none> and password file <none>
I20260807 11:26:26.367408 31624 fs_manager.cc:359] Metadata directory not provided
I20260807 11:26:26.367579 31624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:26:26.367892 31624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:26:26.370858 31624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-2/data/instance:
uuid: "95f75c671a86479dbf9db7c9efe17291"
format_stamp: "Formatted at 2026-08-07 11:26:26 on dist-test-slave-2w3w"
server_key: "22568cd41ab5670ef3f58a98ce90af43"
server_key_iv: "34125b9decdbdf89fec732ec3d0411ae"
server_key_version: "encryptionkey@0"
I20260807 11:26:26.371745 31624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-2/wal/instance:
uuid: "95f75c671a86479dbf9db7c9efe17291"
format_stamp: "Formatted at 2026-08-07 11:26:26 on dist-test-slave-2w3w"
server_key: "22568cd41ab5670ef3f58a98ce90af43"
server_key_iv: "34125b9decdbdf89fec732ec3d0411ae"
server_key_version: "encryptionkey@0"
I20260807 11:26:26.377223 31624 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 11:26:26.380951 31642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:26.382354 31624 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 11:26:26.382500 31624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-2/wal
uuid: "95f75c671a86479dbf9db7c9efe17291"
format_stamp: "Formatted at 2026-08-07 11:26:26 on dist-test-slave-2w3w"
server_key: "22568cd41ab5670ef3f58a98ce90af43"
server_key_iv: "34125b9decdbdf89fec732ec3d0411ae"
server_key_version: "encryptionkey@0"
I20260807 11:26:26.382671 31624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:26:26.401492 31624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:26:26.406203 31624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:26:26.406461 31624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:26:26.425179 31624 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.60:34373
I20260807 11:26:26.425212 31694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.60:34373 every 8 connection(s)
I20260807 11:26:26.426581 31624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-2/data/info.pb
I20260807 11:26:26.429092 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 31624
I20260807 11:26:26.429327 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786101765888975-17172-0/minicluster-data/master-2/wal/instance
I20260807 11:26:26.430029 17172 external_mini_cluster.cc:1428] Setting key 087ca6fe309f4d24d9dfa0b2e4ba8569
I20260807 11:26:26.431195 31695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:26:26.441366 31695 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45723 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 11:26:26 dist-test-slave-2w3w krb5kdc[31455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.60@KRBTEST.COM
Aug 07 11:26:26 dist-test-slave-2w3w krb5kdc[31455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101986, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.60@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:26:26.455461 31698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.432038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52838 (local address 127.16.197.60:34373)
0807 11:26:26.432857 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:26.432872 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:26.432925 (+ 53us) server_negotiation.cc:407] Connection header received
0807 11:26:26.434881 (+ 1956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:26.434911 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:26.435471 (+ 560us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:26.435913 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:26.437278 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.439406 (+ 2128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.440795 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.441856 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.442056 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:26.447179 (+ 5123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:26.447186 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:26.447196 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:26.447236 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:26.450906 (+ 3670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:26.451920 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:26.451926 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:26.451932 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:26.452102 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:26.452558 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:26.452561 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:26.452563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:26.453069 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:26.453336 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:26.453783 (+ 447us) server_negotiation.cc:299] Negotiation successful
0807 11:26:26.454152 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":347,"threads_started":1}
I20260807 11:26:26.460654 31699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.444331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47296 (local address 127.16.197.62:45723)
0807 11:26:26.444838 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:26.444846 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:26.445421 (+ 575us) server_negotiation.cc:407] Connection header received
0807 11:26:26.445626 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:26.445630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:26.445707 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:26.445793 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:26.447353 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.448411 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.449837 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.450483 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.450623 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:26.454296 (+ 3673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:26.454302 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:26.454305 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:26.454350 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:26.457263 (+ 2913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:26.458121 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:26.458130 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:26.458134 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:26.458271 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:26.459056 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:26.459060 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:26.459062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:26.459372 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:26.459547 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:26.460030 (+ 483us) server_negotiation.cc:299] Negotiation successful
0807 11:26:26.460243 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":262,"threads_started":1}
I20260807 11:26:26.460665 31700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.444304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45723 (local address 127.0.0.1:47296)
0807 11:26:26.445242 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:26.445300 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:26.445476 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:26.445903 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:26.445910 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:26.446745 (+ 835us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:26:26.447184 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:26.447199 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.448589 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.448593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:26.449678 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:26.449691 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.450673 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.450677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:26.450799 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:26.451429 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:26:26.451456 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:26:26.454065 (+ 2609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:26:26.457407 (+ 3342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:26.457421 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:26.457427 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:26.457996 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:26.458390 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:26.458395 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:26.458398 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:26.458570 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:26.459512 (+ 942us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:26:26.459521 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:26:26.459720 (+ 199us) client_negotiation.cc:769] Sending connection context
0807 11:26:26.459922 (+ 202us) client_negotiation.cc:240] Negotiation successful
0807 11:26:26.460241 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"thread_start_us":511,"threads_started":1}
I20260807 11:26:26.462100 31695 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.61" port: 33985 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 11:26:26 dist-test-slave-2w3w krb5kdc[31455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
Aug 07 11:26:26 dist-test-slave-2w3w krb5kdc[31455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101986, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.60@KRBTEST.COM for kudu/127.16.197.61@KRBTEST.COM
I20260807 11:26:26.475724 31701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.458612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47312 (local address 127.16.197.62:45723)
0807 11:26:26.459296 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:26.459300 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:26.459319 (+ 19us) server_negotiation.cc:407] Connection header received
0807 11:26:26.459364 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:26.459367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:26.459417 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:26.459540 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:26.461527 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.462631 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.464308 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.464731 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.464825 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:26.470011 (+ 5186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:26.470019 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:26.470026 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:26.470070 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:26.472856 (+ 2786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:26.473644 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:26.473648 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:26.473650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:26.473779 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:26.474187 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:26.474193 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:26.474196 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:26.474588 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:26.474834 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:26.475195 (+ 361us) server_negotiation.cc:299] Negotiation successful
0807 11:26:26.475368 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":377,"threads_started":1}
I20260807 11:26:26.476567 31700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.462904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:33985 (local address 127.0.0.1:59536)
0807 11:26:26.463110 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:26.463127 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:26.463211 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:26.463554 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:26.463557 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:26.463907 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:26:26.464042 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:26.464049 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.465113 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.465116 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:26.465989 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:26.466000 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.467052 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.467057 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:26.467287 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:26.467894 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:26:26.467911 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:26:26.470262 (+ 2351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:26:26.473890 (+ 3628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:26.473898 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:26.473903 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:26.474390 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:26.474800 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:26.474804 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:26.474808 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:26.475015 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:26.475623 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:26:26.475629 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:26:26.475813 (+ 184us) client_negotiation.cc:769] Sending connection context
0807 11:26:26.475982 (+ 169us) client_negotiation.cc:240] Negotiation successful
0807 11:26:26.476266 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260807 11:26:26.476733 31622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.462941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59536 (local address 127.16.197.61:33985)
0807 11:26:26.463162 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:26.463168 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:26.463187 (+ 19us) server_negotiation.cc:407] Connection header received
0807 11:26:26.463329 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:26.463335 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:26.463418 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:26.463521 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:26.464188 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.464953 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.466139 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.466845 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.467020 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:26.470450 (+ 3430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:26.470455 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:26.470458 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:26.470491 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:26.473761 (+ 3270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:26.474507 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:26.474511 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:26.474513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:26.474664 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:26.475139 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:26.475144 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:26.475147 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:26.475486 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:26.475646 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:26.476113 (+ 467us) server_negotiation.cc:299] Negotiation successful
0807 11:26:26.476328 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260807 11:26:26.477659 31695 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 11:26:26 dist-test-slave-2w3w krb5kdc[31455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101986, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.60@KRBTEST.COM for kudu/127.16.197.60@KRBTEST.COM
Aug 07 11:26:26 dist-test-slave-2w3w krb5kdc[31455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.61@KRBTEST.COM
I20260807 11:26:26.491372 31700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.478507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:34373 (local address 127.0.0.1:52854)
0807 11:26:26.478662 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:26.478682 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:26.478794 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:26.479155 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:26.479159 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:26.479541 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:26:26.479663 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:26.479671 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.480610 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.480618 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:26.481459 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:26.481470 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.482168 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.482174 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:26.482354 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:26.482934 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:26:26.482956 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:26:26.485296 (+ 2340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:26:26.488753 (+ 3457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:26.488761 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:26.488766 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:26.489256 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:26.489614 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:26.489619 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:26.489625 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:26.489808 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:26.490312 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:26:26.490316 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:26:26.490457 (+ 141us) client_negotiation.cc:769] Sending connection context
0807 11:26:26.490602 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 11:26:26.490924 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 11:26:26.491464 31698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.478458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52854 (local address 127.16.197.60:34373)
0807 11:26:26.478607 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:26.478613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:26.478735 (+ 122us) server_negotiation.cc:407] Connection header received
0807 11:26:26.478908 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:26.478913 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:26.479044 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:26.479137 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:26.479808 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.480488 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.481593 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.482013 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.482152 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:26.485490 (+ 3338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:26.485498 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:26.485504 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:26.485545 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:26.488611 (+ 3066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:26.489380 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:26.489384 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:26.489386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:26.489504 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:26.489923 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:26.489929 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:26.489932 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:26.490192 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:26.490350 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:26.490675 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 11:26:26.491106 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":161,"server-negotiator.queue_time_us":47}
I20260807 11:26:26.493999 31622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.477973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59542 (local address 127.16.197.61:33985)
0807 11:26:26.478134 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:26.478139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:26.478162 (+ 23us) server_negotiation.cc:407] Connection header received
0807 11:26:26.478318 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:26.478326 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:26.478629 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:26.478725 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:26.480016 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.480738 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.481998 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.482457 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.482594 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:26.488298 (+ 5704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:26.488303 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:26.488306 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:26.488336 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:26.491146 (+ 2810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:26.492007 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:26.492014 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:26.492017 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:26.492169 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:26.492554 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:26.492560 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:26.492563 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:26.492916 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:26.493199 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:26.493421 (+ 222us) server_negotiation.cc:299] Negotiation successful
0807 11:26:26.493661 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 11:26:26.497202 31695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291: Bootstrap starting.
I20260807 11:26:26.499004 17172 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 11:26:26.499505 31540 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } attempt: 4
I20260807 11:26:26.501358 31695 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291: Neither blocks nor log segments found. Creating new log.
I20260807 11:26:26.502794 31695 log.cc:824] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291: Log is configured to *not* fsync() on all Append() calls
I20260807 11:26:26.506619 31695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291: No bootstrap required, opened a new log
Aug 07 11:26:26 dist-test-slave-2w3w krb5kdc[31455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101985, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for kudu/127.16.197.60@KRBTEST.COM
I20260807 11:26:26.517616 31542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.500086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:34373 (local address 127.0.0.1:52860)
0807 11:26:26.500306 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:26.500326 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:26.500435 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:26.501045 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:26.501052 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:26.501632 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:26:26.501879 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:26.501892 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.503564 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.503570 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:26.504661 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:26.504678 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.505744 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.505750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:26.505953 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:26.506726 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:26:26.506749 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:26:26.509751 (+ 3002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:26:26.514385 (+ 4634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:26.514392 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:26.514398 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:26.515069 (+ 671us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:26.515545 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:26.515551 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:26.515555 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:26.515720 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:26.516491 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:26:26.516498 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:26:26.516691 (+ 193us) client_negotiation.cc:769] Sending connection context
0807 11:26:26.516834 (+ 143us) client_negotiation.cc:240] Negotiation successful
0807 11:26:26.517141 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 11:26:26.518679 31698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.500540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52860 (local address 127.16.197.60:34373)
0807 11:26:26.500731 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:26.500737 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:26.500754 (+ 17us) server_negotiation.cc:407] Connection header received
0807 11:26:26.500803 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:26.500809 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:26.500875 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:26.500972 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:26.502154 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.503368 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.504859 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.505554 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.506294 (+ 740us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:26.509989 (+ 3695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:26.509997 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:26.510004 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:26.510054 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:26.514250 (+ 4196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:26.515221 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:26.515227 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:26.515231 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:26.515400 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:26.515853 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:26.515858 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:26.515862 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:26.516245 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:26.516424 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:26.518067 (+ 1643us) server_negotiation.cc:299] Negotiation successful
0807 11:26:26.518284 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 11:26:26.521426 31695 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86a6a8cf64f4752af989c53294584f7" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45723 } } peers { permanent_uuid: "d7c9f1e595fd472e9c15963cdad9d00a" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 33985 } } peers { permanent_uuid: "95f75c671a86479dbf9db7c9efe17291" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } }
I20260807 11:26:26.521876 31695 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:26:26.522013 31695 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95f75c671a86479dbf9db7c9efe17291, State: Initialized, Role: FOLLOWER
I20260807 11:26:26.524652 31540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d86a6a8cf64f4752af989c53294584f7: Bootstrap starting.
I20260807 11:26:26.526551 31695 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86a6a8cf64f4752af989c53294584f7" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45723 } } peers { permanent_uuid: "d7c9f1e595fd472e9c15963cdad9d00a" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 33985 } } peers { permanent_uuid: "95f75c671a86479dbf9db7c9efe17291" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } }
I20260807 11:26:26.529096 31540 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d86a6a8cf64f4752af989c53294584f7: Neither blocks nor log segments found. Creating new log.
I20260807 11:26:26.530614 31540 log.cc:824] T 00000000000000000000000000000000 P d86a6a8cf64f4752af989c53294584f7: Log is configured to *not* fsync() on all Append() calls
I20260807 11:26:26.533193 31622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.511574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59554 (local address 127.16.197.61:33985)
0807 11:26:26.511790 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:26.511796 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:26.512814 (+ 1018us) server_negotiation.cc:407] Connection header received
0807 11:26:26.513379 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:26.513385 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:26.513469 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:26.513546 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:26.516099 (+ 2553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.517294 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.520344 (+ 3050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.521057 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.521156 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:26.525423 (+ 4267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:26.525432 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:26.525436 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:26.525480 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:26.529385 (+ 3905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:26.530527 (+ 1142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:26.530535 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:26.530538 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:26.530713 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:26.531300 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:26.531305 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:26.531309 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:26.531729 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:26.531919 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:26.532592 (+ 673us) server_negotiation.cc:299] Negotiation successful
0807 11:26:26.532834 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 11:26:26.536147 31695 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:26:26.533981 31711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86a6a8cf64f4752af989c53294584f7" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45723 } } peers { permanent_uuid: "d7c9f1e595fd472e9c15963cdad9d00a" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 33985 } } peers { permanent_uuid: "95f75c671a86479dbf9db7c9efe17291" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } } }
I20260807 11:26:26.536924 31711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 11:26:26.537060 31710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.512118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52870 (local address 127.16.197.60:34373)
0807 11:26:26.514825 (+ 2707us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:26.514832 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:26.517731 (+ 2899us) server_negotiation.cc:407] Connection header received
0807 11:26:26.518080 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:26.518086 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:26.518159 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:26.518233 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:26.520772 (+ 2539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.521998 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.524482 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.525177 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.525276 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:26.529622 (+ 4346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:26.529628 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:26.529636 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:26.529682 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:26.533576 (+ 3894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:26.534528 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:26.534533 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:26.534536 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:26.534691 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:26.535200 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:26.535204 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:26.535207 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:26.535608 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:26.535862 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:26.536413 (+ 551us) server_negotiation.cc:299] Negotiation successful
0807 11:26:26.536627 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2512,"thread_start_us":2851,"threads_started":1}
I20260807 11:26:26.537819 31701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.511227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47326 (local address 127.16.197.62:45723)
0807 11:26:26.511427 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:26.511435 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:26.512978 (+ 1543us) server_negotiation.cc:407] Connection header received
0807 11:26:26.513186 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:26.513193 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:26.513289 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:26.513409 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:26.517269 (+ 3860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.518278 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.524325 (+ 6047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.525019 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.525120 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:26.529725 (+ 4605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:26.529731 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:26.529736 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:26.529785 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:26.534116 (+ 4331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:26.535100 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:26.535107 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:26.535110 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:26.535257 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:26.535716 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:26.535721 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:26.535724 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:26.536155 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:26.536466 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:26.537086 (+ 620us) server_negotiation.cc:299] Negotiation successful
0807 11:26:26.537359 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 11:26:26.540360 31540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d86a6a8cf64f4752af989c53294584f7: No bootstrap required, opened a new log
I20260807 11:26:26.551143 31540 raft_consensus.cc:348] T 00000000000000000000000000000000 P d86a6a8cf64f4752af989c53294584f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86a6a8cf64f4752af989c53294584f7" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45723 } } peers { permanent_uuid: "d7c9f1e595fd472e9c15963cdad9d00a" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 33985 } } peers { permanent_uuid: "95f75c671a86479dbf9db7c9efe17291" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } }
I20260807 11:26:26.551644 31540 raft_consensus.cc:374] T 00000000000000000000000000000000 P d86a6a8cf64f4752af989c53294584f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:26:26.551784 31540 raft_consensus.cc:729] T 00000000000000000000000000000000 P d86a6a8cf64f4752af989c53294584f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d86a6a8cf64f4752af989c53294584f7, State: Initialized, Role: FOLLOWER
I20260807 11:26:26.552400 31540 consensus_queue.cc:260] T 00000000000000000000000000000000 P d86a6a8cf64f4752af989c53294584f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86a6a8cf64f4752af989c53294584f7" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45723 } } peers { permanent_uuid: "d7c9f1e595fd472e9c15963cdad9d00a" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 33985 } } peers { permanent_uuid: "95f75c671a86479dbf9db7c9efe17291" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } }
I20260807 11:26:26.553344 31724 sys_catalog.cc:455] T 00000000000000000000000000000000 P d86a6a8cf64f4752af989c53294584f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86a6a8cf64f4752af989c53294584f7" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45723 } } peers { permanent_uuid: "d7c9f1e595fd472e9c15963cdad9d00a" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 33985 } } peers { permanent_uuid: "95f75c671a86479dbf9db7c9efe17291" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } } }
I20260807 11:26:26.553745 31724 sys_catalog.cc:458] T 00000000000000000000000000000000 P d86a6a8cf64f4752af989c53294584f7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 11:26:26.554773 31540 sys_catalog.cc:564] T 00000000000000000000000000000000 P d86a6a8cf64f4752af989c53294584f7 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 11:26:26.565963 31723 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 11:26:26.566228 31723 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 11:26:26.555155 31624 master_runner.cc:186] Error getting master registration for 127.16.197.62:45723: OK, instance_id { permanent_uuid: "d86a6a8cf64f4752af989c53294584f7" instance_seqno: 1786101985352402 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 11:26:26.570926 31624 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 11:26:26.576336 31738 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d86a6a8cf64f4752af989c53294584f7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 11:26:26.576597 31738 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 11:26:26.575248 31468 master_runner.cc:186] Error getting master registration for 127.16.197.61:33985: OK, instance_id { permanent_uuid: "d7c9f1e595fd472e9c15963cdad9d00a" instance_seqno: 1786101985891009 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 11:26:26.587284 31468 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 11:26:26.591379 31491 tablet.cc:2369] T 00000000000000000000000000000000 P d86a6a8cf64f4752af989c53294584f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:26:26.602880 31711 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:26:26.603317 31711 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86a6a8cf64f4752af989c53294584f7" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45723 } } peers { permanent_uuid: "d7c9f1e595fd472e9c15963cdad9d00a" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 33985 } } peers { permanent_uuid: "95f75c671a86479dbf9db7c9efe17291" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } }
I20260807 11:26:26.605237 31711 leader_election.cc:288] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d86a6a8cf64f4752af989c53294584f7 (127.16.197.62:45723), d7c9f1e595fd472e9c15963cdad9d00a (127.16.197.61:33985)
I20260807 11:26:26.606477 31513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "95f75c671a86479dbf9db7c9efe17291" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86a6a8cf64f4752af989c53294584f7" is_pre_election: true
I20260807 11:26:26.607204 31513 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d86a6a8cf64f4752af989c53294584f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95f75c671a86479dbf9db7c9efe17291 in term 0.
I20260807 11:26:26.607896 31646 leader_election.cc:302] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95f75c671a86479dbf9db7c9efe17291, d86a6a8cf64f4752af989c53294584f7; no voters:
I20260807 11:26:26.608457 31711 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:26:26.608613 31711 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:26:26.608721 31711 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:26:26.612684 31711 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86a6a8cf64f4752af989c53294584f7" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45723 } } peers { permanent_uuid: "d7c9f1e595fd472e9c15963cdad9d00a" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 33985 } } peers { permanent_uuid: "95f75c671a86479dbf9db7c9efe17291" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } }
I20260807 11:26:26.614156 31513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "95f75c671a86479dbf9db7c9efe17291" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d86a6a8cf64f4752af989c53294584f7"
I20260807 11:26:26.614403 31513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d86a6a8cf64f4752af989c53294584f7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:26:26.614568 31711 leader_election.cc:288] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291 [CANDIDATE]: Term 1 election: Requested vote from peers d86a6a8cf64f4752af989c53294584f7 (127.16.197.62:45723), d7c9f1e595fd472e9c15963cdad9d00a (127.16.197.61:33985)
I20260807 11:26:26.618166 31513 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d86a6a8cf64f4752af989c53294584f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95f75c671a86479dbf9db7c9efe17291 in term 1.
I20260807 11:26:26.618644 31646 leader_election.cc:302] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95f75c671a86479dbf9db7c9efe17291, d86a6a8cf64f4752af989c53294584f7; no voters:
I20260807 11:26:26.619062 31711 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:26:26.625679 31616 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } attempt: 5
I20260807 11:26:26.627943 31711 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291 [term 1 LEADER]: Becoming Leader. State: Replica: 95f75c671a86479dbf9db7c9efe17291, State: Running, Role: LEADER
I20260807 11:26:26.628641 31711 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86a6a8cf64f4752af989c53294584f7" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45723 } } peers { permanent_uuid: "d7c9f1e595fd472e9c15963cdad9d00a" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 33985 } } peers { permanent_uuid: "95f75c671a86479dbf9db7c9efe17291" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } }
Aug 07 11:26:26 dist-test-slave-2w3w krb5kdc[31455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101985, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.61@KRBTEST.COM for kudu/127.16.197.60@KRBTEST.COM
I20260807 11:26:26.638616 31740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95f75c671a86479dbf9db7c9efe17291. Latest consensus state: current_term: 1 leader_uuid: "95f75c671a86479dbf9db7c9efe17291" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86a6a8cf64f4752af989c53294584f7" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45723 } } peers { permanent_uuid: "d7c9f1e595fd472e9c15963cdad9d00a" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 33985 } } peers { permanent_uuid: "95f75c671a86479dbf9db7c9efe17291" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } } }
I20260807 11:26:26.639184 31740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291 [sys.catalog]: This master's current role is: LEADER
I20260807 11:26:26.647783 31742 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:26:26.649673 31620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.626271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:34373 (local address 127.0.0.1:52874)
0807 11:26:26.626641 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:26.626666 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:26.626792 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:26.627753 (+ 961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:26.627759 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:26.628317 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:26:26.628534 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:26.628546 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.630172 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.630177 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:26.631676 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:26.631693 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.632987 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.632993 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:26.633227 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:26.634012 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:26:26.634038 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:26:26.637639 (+ 3601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:26:26.644399 (+ 6760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:26.644405 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:26.644410 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:26.645465 (+ 1055us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:26.646374 (+ 909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:26.646380 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:26.646384 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:26.646697 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:26.647882 (+ 1185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:26:26.647913 (+ 31us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:26:26.648184 (+ 271us) client_negotiation.cc:769] Sending connection context
0807 11:26:26.648436 (+ 252us) client_negotiation.cc:240] Negotiation successful
0807 11:26:26.649022 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235}
I20260807 11:26:26.652621 31710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:26.627041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52874 (local address 127.16.197.60:34373)
0807 11:26:26.627248 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:26.627254 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:26.627272 (+ 18us) server_negotiation.cc:407] Connection header received
0807 11:26:26.627352 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:26.627357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:26.627436 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:26.627539 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:26.628774 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.629910 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.631904 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:26.632615 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:26.632737 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:26.637938 (+ 5201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:26.637945 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:26.637950 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:26.637989 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:26.644106 (+ 6117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:26.645779 (+ 1673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:26.645785 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:26.645821 (+ 36us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:26.646163 (+ 342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:26.646873 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:26.646878 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:26.646882 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:26.647638 (+ 756us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:26.651161 (+ 3523us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:26.651484 (+ 323us) server_negotiation.cc:299] Negotiation successful
0807 11:26:26.651940 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260807 11:26:26.658535 31648 tablet.cc:2369] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:26:26.658993 31616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7c9f1e595fd472e9c15963cdad9d00a: Bootstrap starting.
I20260807 11:26:26.663288 31616 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7c9f1e595fd472e9c15963cdad9d00a: Neither blocks nor log segments found. Creating new log.
I20260807 11:26:26.664011 31742 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:26:26.668447 31616 log.cc:824] T 00000000000000000000000000000000 P d7c9f1e595fd472e9c15963cdad9d00a: Log is configured to *not* fsync() on all Append() calls
I20260807 11:26:26.673419 31616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7c9f1e595fd472e9c15963cdad9d00a: No bootstrap required, opened a new log
I20260807 11:26:26.682786 31513 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d86a6a8cf64f4752af989c53294584f7 [term 1 FOLLOWER]: Refusing update from remote peer 95f75c671a86479dbf9db7c9efe17291: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:26:26.684078 31740 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d86a6a8cf64f4752af989c53294584f7" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:26:26.688436 31616 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7c9f1e595fd472e9c15963cdad9d00a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86a6a8cf64f4752af989c53294584f7" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45723 } } peers { permanent_uuid: "d7c9f1e595fd472e9c15963cdad9d00a" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 33985 } } peers { permanent_uuid: "95f75c671a86479dbf9db7c9efe17291" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } }
I20260807 11:26:26.689033 31616 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7c9f1e595fd472e9c15963cdad9d00a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:26:26.689200 31616 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7c9f1e595fd472e9c15963cdad9d00a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7c9f1e595fd472e9c15963cdad9d00a, State: Initialized, Role: FOLLOWER
I20260807 11:26:26.690117 31616 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7c9f1e595fd472e9c15963cdad9d00a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86a6a8cf64f4752af989c53294584f7" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45723 } } peers { permanent_uuid: "d7c9f1e595fd472e9c15963cdad9d00a" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 33985 } } peers { permanent_uuid: "95f75c671a86479dbf9db7c9efe17291" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } }
I20260807 11:26:26.697925 31746 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7c9f1e595fd472e9c15963cdad9d00a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86a6a8cf64f4752af989c53294584f7" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45723 } } peers { permanent_uuid: "d7c9f1e595fd472e9c15963cdad9d00a" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 33985 } } peers { permanent_uuid: "95f75c671a86479dbf9db7c9efe17291" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } } }
I20260807 11:26:26.700827 31746 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7c9f1e595fd472e9c15963cdad9d00a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 11:26:26.700753 31616 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7c9f1e595fd472e9c15963cdad9d00a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:26:26.700719 31590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "95f75c671a86479dbf9db7c9efe17291" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7c9f1e595fd472e9c15963cdad9d00a" is_pre_election: true
I20260807 11:26:26.701148 31589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "95f75c671a86479dbf9db7c9efe17291" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7c9f1e595fd472e9c15963cdad9d00a"
I20260807 11:26:26.701469 31590 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d7c9f1e595fd472e9c15963cdad9d00a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95f75c671a86479dbf9db7c9efe17291 in term 0.
I20260807 11:26:26.702113 31588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7c9f1e595fd472e9c15963cdad9d00a [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:26:26.706918 31588 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d7c9f1e595fd472e9c15963cdad9d00a [term 1 FOLLOWER]: Refusing update from remote peer 95f75c671a86479dbf9db7c9efe17291: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:26:26.709764 31740 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7c9f1e595fd472e9c15963cdad9d00a" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 33985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:26:26.735742 31746 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7c9f1e595fd472e9c15963cdad9d00a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95f75c671a86479dbf9db7c9efe17291. Latest consensus state: current_term: 1 leader_uuid: "95f75c671a86479dbf9db7c9efe17291" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86a6a8cf64f4752af989c53294584f7" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45723 } } peers { permanent_uuid: "d7c9f1e595fd472e9c15963cdad9d00a" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 33985 } } peers { permanent_uuid: "95f75c671a86479dbf9db7c9efe17291" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } } }
I20260807 11:26:26.736089 31746 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7c9f1e595fd472e9c15963cdad9d00a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 11:26:26.738093 31744 mvcc.cc:205] Tried to move back new op lower bound from 7315873737423540224 to 7315873737237618688. Current Snapshot: MvccSnapshot[applied={T|T < 7315873737423540224}]
I20260807 11:26:26.746055 31740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "95f75c671a86479dbf9db7c9efe17291" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86a6a8cf64f4752af989c53294584f7" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45723 } } peers { permanent_uuid: "d7c9f1e595fd472e9c15963cdad9d00a" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 33985 } } peers { permanent_uuid: "95f75c671a86479dbf9db7c9efe17291" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } } }
I20260807 11:26:26.746352 31740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291 [sys.catalog]: This master's current role is: LEADER
I20260807 11:26:26.747504 31746 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7c9f1e595fd472e9c15963cdad9d00a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "95f75c671a86479dbf9db7c9efe17291" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86a6a8cf64f4752af989c53294584f7" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45723 } } peers { permanent_uuid: "d7c9f1e595fd472e9c15963cdad9d00a" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 33985 } } peers { permanent_uuid: "95f75c671a86479dbf9db7c9efe17291" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } } }
I20260807 11:26:26.747839 31746 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7c9f1e595fd472e9c15963cdad9d00a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 11:26:26.749006 31749 mvcc.cc:205] Tried to move back new op lower bound from 7315873737423540224 to 7315873737237618688. Current Snapshot: MvccSnapshot[applied={T|T < 7315873737423540224}]
I20260807 11:26:26.752182 31742 catalog_manager.cc:1347] Generated new cluster ID: b3e29eff0a4a4386a87940344186d47f
I20260807 11:26:26.752321 31742 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:26:26.756129 31724 sys_catalog.cc:455] T 00000000000000000000000000000000 P d86a6a8cf64f4752af989c53294584f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95f75c671a86479dbf9db7c9efe17291. Latest consensus state: current_term: 1 leader_uuid: "95f75c671a86479dbf9db7c9efe17291" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86a6a8cf64f4752af989c53294584f7" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45723 } } peers { permanent_uuid: "d7c9f1e595fd472e9c15963cdad9d00a" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 33985 } } peers { permanent_uuid: "95f75c671a86479dbf9db7c9efe17291" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } } }
I20260807 11:26:26.756475 31724 sys_catalog.cc:458] T 00000000000000000000000000000000 P d86a6a8cf64f4752af989c53294584f7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 11:26:26.761212 31711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "95f75c671a86479dbf9db7c9efe17291" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86a6a8cf64f4752af989c53294584f7" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45723 } } peers { permanent_uuid: "d7c9f1e595fd472e9c15963cdad9d00a" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 33985 } } peers { permanent_uuid: "95f75c671a86479dbf9db7c9efe17291" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } } }
I20260807 11:26:26.761633 31711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291 [sys.catalog]: This master's current role is: LEADER
I20260807 11:26:26.776408 31724 sys_catalog.cc:455] T 00000000000000000000000000000000 P d86a6a8cf64f4752af989c53294584f7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "95f75c671a86479dbf9db7c9efe17291" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d86a6a8cf64f4752af989c53294584f7" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45723 } } peers { permanent_uuid: "d7c9f1e595fd472e9c15963cdad9d00a" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 33985 } } peers { permanent_uuid: "95f75c671a86479dbf9db7c9efe17291" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 34373 } } }
I20260807 11:26:26.776691 31724 sys_catalog.cc:458] T 00000000000000000000000000000000 P d86a6a8cf64f4752af989c53294584f7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 11:26:26.822533 31742 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:26:26.825138 31742 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:26:26.868206 31764 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 11:26:26.875389 31742 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95f75c671a86479dbf9db7c9efe17291: Generated new TSK 0
I20260807 11:26:26.876564 31742 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:26:27.581074 31738 catalog_manager.cc:1259] Loaded cluster ID: b3e29eff0a4a4386a87940344186d47f
I20260807 11:26:27.581183 31738 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d86a6a8cf64f4752af989c53294584f7: loading cluster ID for follower catalog manager: success
I20260807 11:26:27.583757 31738 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d86a6a8cf64f4752af989c53294584f7: acquiring CA information for follower catalog manager: success
I20260807 11:26:27.585620 31738 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d86a6a8cf64f4752af989c53294584f7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 11:26:27.873282 31764 catalog_manager.cc:1259] Loaded cluster ID: b3e29eff0a4a4386a87940344186d47f
I20260807 11:26:27.873400 31764 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d7c9f1e595fd472e9c15963cdad9d00a: loading cluster ID for follower catalog manager: success
I20260807 11:26:27.876300 31764 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d7c9f1e595fd472e9c15963cdad9d00a: acquiring CA information for follower catalog manager: success
I20260807 11:26:27.878211 31764 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d7c9f1e595fd472e9c15963cdad9d00a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 11:26:28.007805 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 31468
I20260807 11:26:28.009227 31534 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:26:28.124279 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 31468
I20260807 11:26:28.148590 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 31547
I20260807 11:26:28.149937 31611 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:26:28.265017 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 31547
I20260807 11:26:28.287407 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 31624
I20260807 11:26:28.288532 31689 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:26:28.399466 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 31624
2026-08-07T11:26:28Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4932 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:26:28 dist-test-slave-2w3w krb5kdc[31777](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:26:28 dist-test-slave-2w3w krb5kdc[31777](info): set up 2 sockets
Aug 07 11:26:28 dist-test-slave-2w3w krb5kdc[31777](info): commencing operation
krb5kdc: starting...
W20260807 11:26:29.553020 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.003s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:26:29 dist-test-slave-2w3w krb5kdc[31777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:26:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:26:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:26:29.831460 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:46869
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:42339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:46869
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:26:30.181517 31790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:26:30.181915 31790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:26:30.182104 31790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:26:30.189857 31790 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:26:30.190055 31790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:26:30.190188 31790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:26:30.190308 31790 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:26:30.190388 31790 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:26:30.200987 31790 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:42339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:46869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:46869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:26:30.202515 31790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:26:30.203954 31790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:26:30.216607 31795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:30.216641 31798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:30.216583 31796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:30.217983 31790 server_base.cc:1034] running on GCE node
I20260807 11:26:30.218883 31790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:26:30.220520 31790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:26:30.221765 31790 hybrid_clock.cc:648] HybridClock initialized: now 1786101990221723 us; error 61 us; skew 500 ppm
I20260807 11:26:30.222144 31790 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:26:30 dist-test-slave-2w3w krb5kdc[31777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101990, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:26:30.225423 31790 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:26:30.227227 31790 webserver.cc:460] Webserver started at http://127.16.197.62:36261/ using document root <none> and password file <none>
I20260807 11:26:30.227924 31790 fs_manager.cc:359] Metadata directory not provided
I20260807 11:26:30.228034 31790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:26:30.228305 31790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:26:30.231590 31790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "8be9fde19e6042158f0b95298e96cf5f"
format_stamp: "Formatted at 2026-08-07 11:26:30 on dist-test-slave-2w3w"
server_key: "08af59cb65af12f80b85e6b8b1486db5"
server_key_iv: "d9573be1a6b7c694b91a10523cd4e92a"
server_key_version: "encryptionkey@0"
I20260807 11:26:30.232348 31790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "8be9fde19e6042158f0b95298e96cf5f"
format_stamp: "Formatted at 2026-08-07 11:26:30 on dist-test-slave-2w3w"
server_key: "08af59cb65af12f80b85e6b8b1486db5"
server_key_iv: "d9573be1a6b7c694b91a10523cd4e92a"
server_key_version: "encryptionkey@0"
I20260807 11:26:30.237841 31790 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 11:26:30.241506 31805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:30.243106 31790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:26:30.243323 31790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "8be9fde19e6042158f0b95298e96cf5f"
format_stamp: "Formatted at 2026-08-07 11:26:30 on dist-test-slave-2w3w"
server_key: "08af59cb65af12f80b85e6b8b1486db5"
server_key_iv: "d9573be1a6b7c694b91a10523cd4e92a"
server_key_version: "encryptionkey@0"
I20260807 11:26:30.243556 31790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:26:30.258793 31790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:26:30.263428 31790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:26:30.263756 31790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:26:30.283346 31790 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:46869
I20260807 11:26:30.283371 31857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:46869 every 8 connection(s)
I20260807 11:26:30.284770 31790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:26:30.288722 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 31790
I20260807 11:26:30.288978 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:26:30.289023 31858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:26:30.289896 17172 external_mini_cluster.cc:1428] Setting key 228573e14f8538d221afcc929b62479f
I20260807 11:26:30.304646 31858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8be9fde19e6042158f0b95298e96cf5f: Bootstrap starting.
Aug 07 11:26:30 dist-test-slave-2w3w krb5kdc[31777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:26:30.308647 31858 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8be9fde19e6042158f0b95298e96cf5f: Neither blocks nor log segments found. Creating new log.
I20260807 11:26:30.310129 31858 log.cc:824] T 00000000000000000000000000000000 P 8be9fde19e6042158f0b95298e96cf5f: Log is configured to *not* fsync() on all Append() calls
I20260807 11:26:30.313675 31858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8be9fde19e6042158f0b95298e96cf5f: No bootstrap required, opened a new log
I20260807 11:26:30.316366 31861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:30.291910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49270 (local address 127.16.197.62:46869)
0807 11:26:30.292791 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:30.292804 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:30.292846 (+ 42us) server_negotiation.cc:407] Connection header received
0807 11:26:30.294431 (+ 1585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:30.294454 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:30.294879 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:30.295314 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:30.297433 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:30.298734 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:30.300937 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:30.301684 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:30.301831 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:30.307098 (+ 5267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:30.307109 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:30.307128 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:30.307183 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:30.311315 (+ 4132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:30.312196 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:30.312205 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:30.312215 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:30.312460 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:30.312860 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:30.312865 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:30.312868 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:30.313603 (+ 735us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:30.313917 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:30.314406 (+ 489us) server_negotiation.cc:299] Negotiation successful
0807 11:26:30.314887 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":422,"threads_started":1}
I20260807 11:26:30.320498 17172 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 11:26:30.323278 31858 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8be9fde19e6042158f0b95298e96cf5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8be9fde19e6042158f0b95298e96cf5f" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46869 } }
I20260807 11:26:30.323698 31858 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8be9fde19e6042158f0b95298e96cf5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:26:30.323836 31858 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8be9fde19e6042158f0b95298e96cf5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8be9fde19e6042158f0b95298e96cf5f, State: Initialized, Role: FOLLOWER
I20260807 11:26:30.324424 31858 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8be9fde19e6042158f0b95298e96cf5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8be9fde19e6042158f0b95298e96cf5f" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46869 } }
I20260807 11:26:30.324761 31858 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8be9fde19e6042158f0b95298e96cf5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:26:30.324934 31858 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8be9fde19e6042158f0b95298e96cf5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:26:30.325114 31858 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8be9fde19e6042158f0b95298e96cf5f [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:26:30.327457 31858 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8be9fde19e6042158f0b95298e96cf5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8be9fde19e6042158f0b95298e96cf5f" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46869 } }
I20260807 11:26:30.327972 31858 leader_election.cc:302] T 00000000000000000000000000000000 P 8be9fde19e6042158f0b95298e96cf5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8be9fde19e6042158f0b95298e96cf5f; no voters:
I20260807 11:26:30.328588 31858 leader_election.cc:288] T 00000000000000000000000000000000 P 8be9fde19e6042158f0b95298e96cf5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:26:30.328718 31863 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8be9fde19e6042158f0b95298e96cf5f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:26:30.329463 31863 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8be9fde19e6042158f0b95298e96cf5f [term 1 LEADER]: Becoming Leader. State: Replica: 8be9fde19e6042158f0b95298e96cf5f, State: Running, Role: LEADER
I20260807 11:26:30.329977 31863 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8be9fde19e6042158f0b95298e96cf5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8be9fde19e6042158f0b95298e96cf5f" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46869 } }
I20260807 11:26:30.330693 31858 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8be9fde19e6042158f0b95298e96cf5f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:26:30.333089 31866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8be9fde19e6042158f0b95298e96cf5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8be9fde19e6042158f0b95298e96cf5f. Latest consensus state: current_term: 1 leader_uuid: "8be9fde19e6042158f0b95298e96cf5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8be9fde19e6042158f0b95298e96cf5f" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46869 } } }
I20260807 11:26:30.333602 31866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8be9fde19e6042158f0b95298e96cf5f [sys.catalog]: This master's current role is: LEADER
I20260807 11:26:30.333190 31864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8be9fde19e6042158f0b95298e96cf5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8be9fde19e6042158f0b95298e96cf5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8be9fde19e6042158f0b95298e96cf5f" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46869 } } }
I20260807 11:26:30.336100 31864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8be9fde19e6042158f0b95298e96cf5f [sys.catalog]: This master's current role is: LEADER
I20260807 11:26:30.336320 31873 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:26:30.342525 31873 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:26:30.351722 31873 catalog_manager.cc:1347] Generated new cluster ID: 3e304f344502404faad38ab0c670df54
I20260807 11:26:30.351862 31873 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:26:30.363185 31873 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:26:30.364527 31873 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:26:30.383951 31873 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8be9fde19e6042158f0b95298e96cf5f: Generated new TSK 0
I20260807 11:26:30.384871 31873 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:26:30.433017 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 31790
I20260807 11:26:30.434291 31853 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:26:30.551090 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 31790
2026-08-07T11:26:30Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2144 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:26:30 dist-test-slave-2w3w krb5kdc[31887](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:26:30 dist-test-slave-2w3w krb5kdc[31887](info): set up 2 sockets
Aug 07 11:26:30 dist-test-slave-2w3w krb5kdc[31887](info): commencing operation
krb5kdc: starting...
W20260807 11:26:31.694008 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.003s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:26:31 dist-test-slave-2w3w krb5kdc[31887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:26:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:26:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:26:31.970777 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:42873
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:36135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:42873
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:26:32.311329 31900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:26:32.311686 31900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:26:32.311857 31900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:26:32.319244 31900 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:26:32.319402 31900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:26:32.319525 31900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:26:32.319626 31900 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:26:32.319710 31900 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:26:32.329627 31900 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:36135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:42873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:42873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:26:32.331104 31900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:26:32.332487 31900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:26:32.344550 31905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:32.344722 31908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:32.344755 31906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:32.345575 31900 server_base.cc:1034] running on GCE node
I20260807 11:26:32.346379 31900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:26:32.347892 31900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:26:32.349143 31900 hybrid_clock.cc:648] HybridClock initialized: now 1786101992349109 us; error 50 us; skew 500 ppm
I20260807 11:26:32.349524 31900 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:26:32 dist-test-slave-2w3w krb5kdc[31887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101992, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:26:32.357496 31900 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:26:32.359174 31900 webserver.cc:460] Webserver started at http://127.16.197.62:39769/ using document root <none> and password file <none>
I20260807 11:26:32.359900 31900 fs_manager.cc:359] Metadata directory not provided
I20260807 11:26:32.360064 31900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:26:32.360384 31900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:26:32.363548 31900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "ad8a83ed1c634f0eaf02452b9216f675"
format_stamp: "Formatted at 2026-08-07 11:26:32 on dist-test-slave-2w3w"
server_key: "b9b9d9f56e145c686e6c5d31a63d41ff"
server_key_iv: "31f960df17f3be57b24f0fc8cb28bdc1"
server_key_version: "encryptionkey@0"
I20260807 11:26:32.364357 31900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "ad8a83ed1c634f0eaf02452b9216f675"
format_stamp: "Formatted at 2026-08-07 11:26:32 on dist-test-slave-2w3w"
server_key: "b9b9d9f56e145c686e6c5d31a63d41ff"
server_key_iv: "31f960df17f3be57b24f0fc8cb28bdc1"
server_key_version: "encryptionkey@0"
I20260807 11:26:32.369623 31900 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 11:26:32.373071 31916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:32.374372 31900 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 11:26:32.374557 31900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "ad8a83ed1c634f0eaf02452b9216f675"
format_stamp: "Formatted at 2026-08-07 11:26:32 on dist-test-slave-2w3w"
server_key: "b9b9d9f56e145c686e6c5d31a63d41ff"
server_key_iv: "31f960df17f3be57b24f0fc8cb28bdc1"
server_key_version: "encryptionkey@0"
I20260807 11:26:32.374778 31900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:26:32.394788 31900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:26:32.399369 31900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:26:32.399682 31900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:26:32.418838 31900 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:42873
I20260807 11:26:32.418887 31971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:42873 every 8 connection(s)
I20260807 11:26:32.420293 31900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:26:32.424008 31973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:26:32.427431 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 31900
I20260807 11:26:32.427672 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:26:32.428498 17172 external_mini_cluster.cc:1428] Setting key 9393f3df443e76424446771b8c176bd5
I20260807 11:26:32.437641 31973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675: Bootstrap starting.
I20260807 11:26:32.441637 31973 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675: Neither blocks nor log segments found. Creating new log.
I20260807 11:26:32.443089 31973 log.cc:824] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675: Log is configured to *not* fsync() on all Append() calls
Aug 07 11:26:32 dist-test-slave-2w3w krb5kdc[31887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:26:32.446574 31973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675: No bootstrap required, opened a new log
I20260807 11:26:32.454187 31976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:32.430776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38844 (local address 127.16.197.62:42873)
0807 11:26:32.431639 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:32.431651 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:32.431693 (+ 42us) server_negotiation.cc:407] Connection header received
0807 11:26:32.433385 (+ 1692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:32.433409 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:32.433819 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:32.434174 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:32.436112 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:32.437425 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:32.439274 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:32.440020 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:32.440167 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:32.445503 (+ 5336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:32.445514 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:32.445534 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:32.445584 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:32.449561 (+ 3977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:32.450266 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:32.450275 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:32.450283 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:32.450513 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:32.450888 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:32.450893 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:32.450897 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:32.451634 (+ 737us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:32.451942 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:32.452331 (+ 389us) server_negotiation.cc:299] Negotiation successful
0807 11:26:32.452814 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":443,"threads_started":1}
I20260807 11:26:32.456900 31973 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8a83ed1c634f0eaf02452b9216f675" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42873 } }
I20260807 11:26:32.457270 31973 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:26:32.457378 31973 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad8a83ed1c634f0eaf02452b9216f675, State: Initialized, Role: FOLLOWER
I20260807 11:26:32.457921 31973 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8a83ed1c634f0eaf02452b9216f675" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42873 } }
I20260807 11:26:32.458168 31973 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:26:32.458304 31973 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:26:32.458482 31973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:26:32.460698 31973 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8a83ed1c634f0eaf02452b9216f675" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42873 } }
I20260807 11:26:32.461189 31973 leader_election.cc:302] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad8a83ed1c634f0eaf02452b9216f675; no voters:
I20260807 11:26:32.461751 31973 leader_election.cc:288] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:26:32.461895 31978 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:26:32.462934 31978 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [term 1 LEADER]: Becoming Leader. State: Replica: ad8a83ed1c634f0eaf02452b9216f675, State: Running, Role: LEADER
I20260807 11:26:32.463575 31978 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8a83ed1c634f0eaf02452b9216f675" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42873 } }
I20260807 11:26:32.464040 31973 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:26:32.466580 31979 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad8a83ed1c634f0eaf02452b9216f675" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8a83ed1c634f0eaf02452b9216f675" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42873 } } }
I20260807 11:26:32.466552 31980 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad8a83ed1c634f0eaf02452b9216f675. Latest consensus state: current_term: 1 leader_uuid: "ad8a83ed1c634f0eaf02452b9216f675" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8a83ed1c634f0eaf02452b9216f675" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42873 } } }
I20260807 11:26:32.467056 31979 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [sys.catalog]: This master's current role is: LEADER
I20260807 11:26:32.467064 31980 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [sys.catalog]: This master's current role is: LEADER
I20260807 11:26:32.469208 31987 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:26:32.475019 31987 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:26:32.484226 31987 catalog_manager.cc:1347] Generated new cluster ID: 3d5eb72df9f2420db50028e09f26da9c
I20260807 11:26:32.484349 31987 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:26:32.500802 31987 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:26:32.502355 31987 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:26:32.510848 31987 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675: Generated new TSK 0
I20260807 11:26:32.512064 31987 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.197.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
I20260807 11:26:32.581020 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:0
--local_ip_for_outbound_sockets=127.16.197.1
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:42873
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:36135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:26:32.928373 32001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:26:32.928668 32001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:26:32.928766 32001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:26:32.936344 32001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:26:32.936455 32001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:26:32.936663 32001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:26:32.946717 32001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:36135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:42873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:26:32.948040 32001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:26:32.949420 32001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:26:32.963318 32006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:32.963388 32007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:32.963500 32010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:32.964294 32001 server_base.cc:1034] running on GCE node
I20260807 11:26:32.965036 32001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:26:32.966033 32001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:26:32.967279 32001 hybrid_clock.cc:648] HybridClock initialized: now 1786101992967228 us; error 39 us; skew 500 ppm
I20260807 11:26:32.967643 32001 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:26:32 dist-test-slave-2w3w krb5kdc[31887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101992, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:26:32.971220 32001 init.cc:377] Logged in from keytab as kudu/127.16.197.1@KRBTEST.COM (short username kudu)
I20260807 11:26:32.972877 32001 webserver.cc:460] Webserver started at http://127.16.197.1:35905/ using document root <none> and password file <none>
I20260807 11:26:32.973608 32001 fs_manager.cc:359] Metadata directory not provided
I20260807 11:26:32.973758 32001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:26:32.974081 32001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:26:32.977314 32001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/data/instance:
uuid: "0497e7bcd8254497bb2635405cde23ee"
format_stamp: "Formatted at 2026-08-07 11:26:32 on dist-test-slave-2w3w"
server_key: "21a65431e8ba46ba810568ddc6342b46"
server_key_iv: "f5a48fc07a961cc0679ce8cb02d33032"
server_key_version: "encryptionkey@0"
I20260807 11:26:32.978144 32001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance:
uuid: "0497e7bcd8254497bb2635405cde23ee"
format_stamp: "Formatted at 2026-08-07 11:26:32 on dist-test-slave-2w3w"
server_key: "21a65431e8ba46ba810568ddc6342b46"
server_key_iv: "f5a48fc07a961cc0679ce8cb02d33032"
server_key_version: "encryptionkey@0"
I20260807 11:26:32.983834 32001 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 11:26:32.987531 32017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:32.988993 32001 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 11:26:32.989213 32001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "0497e7bcd8254497bb2635405cde23ee"
format_stamp: "Formatted at 2026-08-07 11:26:32 on dist-test-slave-2w3w"
server_key: "21a65431e8ba46ba810568ddc6342b46"
server_key_iv: "f5a48fc07a961cc0679ce8cb02d33032"
server_key_version: "encryptionkey@0"
I20260807 11:26:32.989449 32001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:26:33.004204 32001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:26:33.008282 32001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:26:33.008607 32001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:26:33.009635 32001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:26:33.011243 32001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:26:33.011345 32001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:33.011500 32001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:26:33.011577 32001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:33.048517 32001 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:45945
I20260807 11:26:33.048573 32135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:45945 every 8 connection(s)
I20260807 11:26:33.050315 32001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:26:33.052611 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 32001
I20260807 11:26:33.052928 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance
I20260807 11:26:33.053787 17172 external_mini_cluster.cc:1428] Setting key 0b8c7e1bc2906c90ab2f42f7ec1e016c
Aug 07 11:26:33 dist-test-slave-2w3w krb5kdc[31887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101992, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:26:33.076359 32140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:33.054795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:35337 (local address 127.16.197.62:42873)
0807 11:26:33.055278 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:33.055284 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:33.055335 (+ 51us) server_negotiation.cc:407] Connection header received
0807 11:26:33.057834 (+ 2499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:33.057837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:33.057906 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:33.057998 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:33.060179 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:33.060891 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:33.063567 (+ 2676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:33.063981 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:33.064141 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:33.068715 (+ 4574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:33.068724 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:33.068731 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:33.068777 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:33.072740 (+ 3963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:33.073568 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:33.073577 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:33.073583 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:33.073742 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:33.074167 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:33.074174 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:33.074179 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:33.074584 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:33.074774 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:33.075565 (+ 791us) server_negotiation.cc:299] Negotiation successful
0807 11:26:33.075846 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":249,"threads_started":1}
I20260807 11:26:33.077045 32139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:33.054203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.1:35337)
0807 11:26:33.055180 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:33.055245 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:33.057536 (+ 2291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:33.058276 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:33.058290 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:33.059154 (+ 864us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:26:33.059985 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:33.060013 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:33.061074 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:33.061083 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:33.063406 (+ 2323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:33.063425 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:33.064240 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:33.064249 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:33.064606 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:33.065434 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:26:33.065478 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:26:33.068474 (+ 2996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:26:33.072910 (+ 4436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:33.072918 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:33.072932 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:33.073392 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:33.073851 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:33.073854 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:33.073856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:33.074048 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:33.074733 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:26:33.074739 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:26:33.075168 (+ 429us) client_negotiation.cc:769] Sending connection context
0807 11:26:33.075405 (+ 237us) client_negotiation.cc:240] Negotiation successful
0807 11:26:33.075841 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":399,"threads_started":1}
I20260807 11:26:33.078780 32136 heartbeater.cc:347] Connected to a master server at 127.16.197.62:42873
I20260807 11:26:33.079126 32136 heartbeater.cc:464] Registering TS with master...
I20260807 11:26:33.079907 32136 heartbeater.cc:511] Master 127.16.197.62:42873 requested a full tablet report, sending...
I20260807 11:26:33.082106 31934 ts_manager.cc:194] Registered new tserver with Master: 0497e7bcd8254497bb2635405cde23ee (127.16.197.1:45945)
I20260807 11:26:33.083735 31934 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.1@KRBTEST.COM'} at 127.16.197.1:35337
WARNING: no policy specified for kudu/127.16.197.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
I20260807 11:26:33.119349 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:0
--local_ip_for_outbound_sockets=127.16.197.2
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:42873
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:36135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:26:33.455502 32143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:26:33.455830 32143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:26:33.456010 32143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:26:33.463469 32143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:26:33.463644 32143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:26:33.463814 32143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:26:33.474249 32143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:36135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:42873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:26:33.475951 32143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:26:33.477274 32143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:26:33.490976 32149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:33.491055 32148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:33.491030 32151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:33.491843 32143 server_base.cc:1034] running on GCE node
I20260807 11:26:33.492455 32143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:26:33.493456 32143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:26:33.494704 32143 hybrid_clock.cc:648] HybridClock initialized: now 1786101993494638 us; error 62 us; skew 500 ppm
I20260807 11:26:33.495121 32143 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:26:33 dist-test-slave-2w3w krb5kdc[31887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101993, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:26:33.498270 32143 init.cc:377] Logged in from keytab as kudu/127.16.197.2@KRBTEST.COM (short username kudu)
I20260807 11:26:33.499989 32143 webserver.cc:460] Webserver started at http://127.16.197.2:38967/ using document root <none> and password file <none>
I20260807 11:26:33.500697 32143 fs_manager.cc:359] Metadata directory not provided
I20260807 11:26:33.500850 32143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:26:33.501179 32143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:26:33.504374 32143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/data/instance:
uuid: "6174176c30e34fd1afb4529ac87df331"
format_stamp: "Formatted at 2026-08-07 11:26:33 on dist-test-slave-2w3w"
server_key: "29c20b461afe09c2275938a988d709c9"
server_key_iv: "b9a21728bf26fbf0bfa0d98598a5bc97"
server_key_version: "encryptionkey@0"
I20260807 11:26:33.505200 32143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance:
uuid: "6174176c30e34fd1afb4529ac87df331"
format_stamp: "Formatted at 2026-08-07 11:26:33 on dist-test-slave-2w3w"
server_key: "29c20b461afe09c2275938a988d709c9"
server_key_iv: "b9a21728bf26fbf0bfa0d98598a5bc97"
server_key_version: "encryptionkey@0"
I20260807 11:26:33.510535 32143 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 11:26:33.514125 32160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:33.515662 32143 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:26:33.515847 32143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "6174176c30e34fd1afb4529ac87df331"
format_stamp: "Formatted at 2026-08-07 11:26:33 on dist-test-slave-2w3w"
server_key: "29c20b461afe09c2275938a988d709c9"
server_key_iv: "b9a21728bf26fbf0bfa0d98598a5bc97"
server_key_version: "encryptionkey@0"
I20260807 11:26:33.516081 32143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:26:33.538039 32143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:26:33.542059 32143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:26:33.542299 32143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:26:33.543267 32143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:26:33.544554 32143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:26:33.544647 32143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:33.544735 32143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:26:33.544780 32143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:33.578423 32143 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:35973
I20260807 11:26:33.578482 32276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:35973 every 8 connection(s)
I20260807 11:26:33.579910 32143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:26:33.585947 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 32143
I20260807 11:26:33.586150 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance
I20260807 11:26:33.586738 17172 external_mini_cluster.cc:1428] Setting key 03e8216c30d423e80d731283a2fd23e3
Aug 07 11:26:33 dist-test-slave-2w3w krb5kdc[31887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101993, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:26:33.601012 32280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:33.582544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:36923 (local address 127.16.197.62:42873)
0807 11:26:33.583075 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:33.583080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:33.583986 (+ 906us) server_negotiation.cc:407] Connection header received
0807 11:26:33.585859 (+ 1873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:33.585865 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:33.585967 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:33.586101 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:33.588102 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:33.588899 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:33.590504 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:33.590951 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:33.591141 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:33.595107 (+ 3966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:33.595111 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:33.595115 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:33.595149 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:33.597748 (+ 2599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:33.598532 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:33.598537 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:33.598539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:33.598672 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:33.599193 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:33.599197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:33.599199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:33.599476 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:33.599638 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:33.600360 (+ 722us) server_negotiation.cc:299] Negotiation successful
0807 11:26:33.600612 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":260,"threads_started":1}
I20260807 11:26:33.602011 32281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:33.582951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.2:36923)
0807 11:26:33.583812 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:33.583856 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:33.585573 (+ 1717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:33.586431 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:33.586445 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:33.587234 (+ 789us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:26:33.587901 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:33.587922 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:33.589073 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:33.589077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:33.590322 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:33.590334 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:33.591251 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:33.591258 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:33.591507 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:33.592233 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:26:33.592257 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:26:33.594819 (+ 2562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:26:33.597914 (+ 3095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:33.597924 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:33.597942 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:33.598385 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:33.598788 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:33.598793 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:33.598797 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:33.599065 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:33.599609 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:26:33.599617 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:26:33.599992 (+ 375us) client_negotiation.cc:769] Sending connection context
0807 11:26:33.600261 (+ 269us) client_negotiation.cc:240] Negotiation successful
0807 11:26:33.600824 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":284,"threads_started":1}
I20260807 11:26:33.603515 32277 heartbeater.cc:347] Connected to a master server at 127.16.197.62:42873
I20260807 11:26:33.603876 32277 heartbeater.cc:464] Registering TS with master...
I20260807 11:26:33.604833 32277 heartbeater.cc:511] Master 127.16.197.62:42873 requested a full tablet report, sending...
I20260807 11:26:33.606527 31934 ts_manager.cc:194] Registered new tserver with Master: 6174176c30e34fd1afb4529ac87df331 (127.16.197.2:35973)
I20260807 11:26:33.607432 31934 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.2@KRBTEST.COM'} at 127.16.197.2:36923
WARNING: no policy specified for kudu/127.16.197.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
I20260807 11:26:33.650857 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:0
--local_ip_for_outbound_sockets=127.16.197.3
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:42873
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:36135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:26:33.992120 32284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:26:33.992413 32284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:26:33.992580 32284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:26:34.000290 32284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:26:34.000438 32284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:26:34.000638 32284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:26:34.010527 32284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:36135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:42873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:26:34.012015 32284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:26:34.013520 32284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:26:34.027369 32293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:34.027535 32290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:34.027323 32291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:34.028714 32284 server_base.cc:1034] running on GCE node
I20260807 11:26:34.029395 32284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:26:34.030316 32284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:26:34.031517 32284 hybrid_clock.cc:648] HybridClock initialized: now 1786101994031458 us; error 57 us; skew 500 ppm
I20260807 11:26:34.031883 32284 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:26:34 dist-test-slave-2w3w krb5kdc[31887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101994, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:26:34.035259 32284 init.cc:377] Logged in from keytab as kudu/127.16.197.3@KRBTEST.COM (short username kudu)
I20260807 11:26:34.036892 32284 webserver.cc:460] Webserver started at http://127.16.197.3:36869/ using document root <none> and password file <none>
I20260807 11:26:34.037590 32284 fs_manager.cc:359] Metadata directory not provided
I20260807 11:26:34.037743 32284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:26:34.038069 32284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:26:34.041100 32284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/data/instance:
uuid: "152888668c1a4d43bdc59dedb3facdd5"
format_stamp: "Formatted at 2026-08-07 11:26:34 on dist-test-slave-2w3w"
server_key: "b9c421176e38932e8ddd7827d62b9be9"
server_key_iv: "712fb01d6d4a437b94f68ce11ee62167"
server_key_version: "encryptionkey@0"
I20260807 11:26:34.041932 32284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance:
uuid: "152888668c1a4d43bdc59dedb3facdd5"
format_stamp: "Formatted at 2026-08-07 11:26:34 on dist-test-slave-2w3w"
server_key: "b9c421176e38932e8ddd7827d62b9be9"
server_key_iv: "712fb01d6d4a437b94f68ce11ee62167"
server_key_version: "encryptionkey@0"
I20260807 11:26:34.047802 32284 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 11:26:34.052249 32301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:34.053620 32284 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.003s
I20260807 11:26:34.053800 32284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "152888668c1a4d43bdc59dedb3facdd5"
format_stamp: "Formatted at 2026-08-07 11:26:34 on dist-test-slave-2w3w"
server_key: "b9c421176e38932e8ddd7827d62b9be9"
server_key_iv: "712fb01d6d4a437b94f68ce11ee62167"
server_key_version: "encryptionkey@0"
I20260807 11:26:34.054011 32284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:26:34.076294 32284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:26:34.080776 32284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:26:34.081096 32284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:26:34.082151 32284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:26:34.083648 32284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:26:34.083736 32284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:34.083890 32284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:26:34.083951 32284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:34.086843 32136 heartbeater.cc:503] Master 127.16.197.62:42873 was elected leader, sending a full tablet report...
I20260807 11:26:34.119663 32284 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:45307
I20260807 11:26:34.119765 32417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:45307 every 8 connection(s)
I20260807 11:26:34.121248 32284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:26:34.128640 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 32284
I20260807 11:26:34.128932 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance
I20260807 11:26:34.129523 17172 external_mini_cluster.cc:1428] Setting key 93ee0b3d4412b904a7f7520dfc01b1c3
Aug 07 11:26:34 dist-test-slave-2w3w krb5kdc[31887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101994, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:26:34.143134 32420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:34.124131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:47785 (local address 127.16.197.62:42873)
0807 11:26:34.124615 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:34.124620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:34.125597 (+ 977us) server_negotiation.cc:407] Connection header received
0807 11:26:34.127518 (+ 1921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:34.127523 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:34.127622 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:34.127785 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:34.129486 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.130683 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.132628 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.133286 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.133401 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:34.137254 (+ 3853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:34.137260 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:34.137263 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:34.137299 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:34.140142 (+ 2843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:34.140849 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:34.140853 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:34.140856 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:34.140977 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:34.141398 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:34.141402 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:34.141404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:34.141689 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:34.141830 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:34.142552 (+ 722us) server_negotiation.cc:299] Negotiation successful
0807 11:26:34.142758 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":238,"threads_started":1}
I20260807 11:26:34.144210 32421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:34.124709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.3:47785)
0807 11:26:34.125426 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:34.125471 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:34.127220 (+ 1749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:34.127965 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:34.127973 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:34.128532 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:26:34.129271 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:34.129291 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.130863 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.130868 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:34.132464 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:34.132480 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.133513 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.133519 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:34.133743 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:34.134482 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:26:34.134507 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:26:34.137001 (+ 2494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:26:34.140279 (+ 3278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:34.140286 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:34.140301 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:34.140733 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:34.141070 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:34.141073 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:34.141075 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:34.141253 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:34.141804 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:26:34.141809 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:26:34.142153 (+ 344us) client_negotiation.cc:769] Sending connection context
0807 11:26:34.142393 (+ 240us) client_negotiation.cc:240] Negotiation successful
0807 11:26:34.142824 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":285,"threads_started":1}
I20260807 11:26:34.145936 32418 heartbeater.cc:347] Connected to a master server at 127.16.197.62:42873
I20260807 11:26:34.146311 32418 heartbeater.cc:464] Registering TS with master...
I20260807 11:26:34.147238 32418 heartbeater.cc:511] Master 127.16.197.62:42873 requested a full tablet report, sending...
I20260807 11:26:34.148856 31933 ts_manager.cc:194] Registered new tserver with Master: 152888668c1a4d43bdc59dedb3facdd5 (127.16.197.3:45307)
I20260807 11:26:34.149766 31933 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.3@KRBTEST.COM'} at 127.16.197.3:47785
I20260807 11:26:34.158574 17172 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 11:26:34.177637 32420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:34.163773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38856 (local address 127.16.197.62:42873)
0807 11:26:34.163993 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:34.163998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:34.164430 (+ 432us) server_negotiation.cc:407] Connection header received
0807 11:26:34.164581 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:34.164585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:34.164656 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:34.164748 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:34.166238 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.167077 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.168580 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.168984 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.169079 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:34.172220 (+ 3141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:34.172224 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:34.172228 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:34.172261 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:34.175289 (+ 3028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:34.175966 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:34.175970 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:34.175972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:34.176095 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:34.176429 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:34.176432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:34.176434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:34.176756 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:34.176936 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:34.177202 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 11:26:34.177381 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 11:26:34.182052 31933 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38856:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 11:26:34.189373 31933 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:26:34.223999 32431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:34.202484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:45945 (local address 127.0.0.1:52532)
0807 11:26:34.214002 (+ 11518us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:34.214022 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:34.214163 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:34.215089 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:34.215097 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:34.215133 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:26:34.215586 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:34.215601 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.218026 (+ 2425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.218033 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:34.220912 (+ 2879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:34.220938 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.222949 (+ 2011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.222989 (+ 40us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:34.223208 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:34.223266 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 11:26:34.223421 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 11:26:34.223624 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11359,"thread_start_us":6390,"threads_started":1}
I20260807 11:26:34.224027 32435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:34.203040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52532 (local address 127.16.197.1:45945)
0807 11:26:34.213602 (+ 10562us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:34.213614 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:34.214311 (+ 697us) server_negotiation.cc:407] Connection header received
0807 11:26:34.214432 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:34.214440 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:34.214776 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:34.215814 (+ 1038us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:26:34.216145 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.217841 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.221169 (+ 3328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.222700 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.222855 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:34.223164 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:34.223491 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 11:26:34.223680 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10305,"thread_start_us":10085,"threads_started":1}
I20260807 11:26:34.225669 32432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:34.202328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:45307 (local address 127.0.0.1:51942)
0807 11:26:34.209447 (+ 7119us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:34.209508 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:34.209766 (+ 258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:34.217720 (+ 7954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:34.217725 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:34.217762 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:26:34.218383 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:34.218397 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.220732 (+ 2335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.220737 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:34.222799 (+ 2062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:34.222818 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.224801 (+ 1983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.224807 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:34.224977 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:34.225011 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 11:26:34.225139 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 11:26:34.225293 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6743,"thread_start_us":11342,"threads_started":1}
I20260807 11:26:34.226006 32434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:34.202582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51942 (local address 127.16.197.3:45307)
0807 11:26:34.217113 (+ 14531us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:34.217122 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:34.217165 (+ 43us) server_negotiation.cc:407] Connection header received
0807 11:26:34.217263 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:34.217271 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:34.217497 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:34.218616 (+ 1119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:26:34.218907 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.220570 (+ 1663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.223049 (+ 2479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.224544 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.224751 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:34.224991 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:34.225488 (+ 497us) server_negotiation.cc:299] Negotiation successful
0807 11:26:34.225675 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14295,"thread_start_us":14145,"threads_started":1}
I20260807 11:26:34.226318 32437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:34.205622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50764 (local address 127.16.197.2:35973)
0807 11:26:34.215706 (+ 10084us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:34.215717 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:34.216440 (+ 723us) server_negotiation.cc:407] Connection header received
0807 11:26:34.216613 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:34.216625 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:34.216913 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:34.217161 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:26:34.218522 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.219904 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.223219 (+ 3315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.224990 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.225148 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:34.225331 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:34.225769 (+ 438us) server_negotiation.cc:299] Negotiation successful
0807 11:26:34.225974 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9893,"thread_start_us":9757,"threads_started":1}
I20260807 11:26:34.226841 32436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:34.205001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:35973 (local address 127.0.0.1:50764)
0807 11:26:34.216277 (+ 11276us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:34.216299 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:34.216438 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:34.217609 (+ 1171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:34.217613 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:34.217643 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:26:34.218246 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:34.218266 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.220106 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.220110 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:34.223012 (+ 2902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:34.223027 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.225217 (+ 2190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.225223 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:34.225386 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:34.225429 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 11:26:34.225517 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 11:26:34.225639 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11080,"mutex_wait_us":1,"thread_start_us":10898,"threads_started":1}
I20260807 11:26:34.228561 32067 tablet_service.cc:1467] Processing CreateTablet for tablet 118435a0a85240d684ffb4098d22cfca (DEFAULT_TABLE table=test-table [id=77797c439fff4e2a9a670791f488007b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:26:34.229863 32067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 118435a0a85240d684ffb4098d22cfca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:26:34.230209 32350 tablet_service.cc:1467] Processing CreateTablet for tablet 118435a0a85240d684ffb4098d22cfca (DEFAULT_TABLE table=test-table [id=77797c439fff4e2a9a670791f488007b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:26:34.230701 32210 tablet_service.cc:1467] Processing CreateTablet for tablet 118435a0a85240d684ffb4098d22cfca (DEFAULT_TABLE table=test-table [id=77797c439fff4e2a9a670791f488007b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:26:34.231664 32350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 118435a0a85240d684ffb4098d22cfca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:26:34.232020 32210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 118435a0a85240d684ffb4098d22cfca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:26:34.241613 32438 tablet_bootstrap.cc:492] T 118435a0a85240d684ffb4098d22cfca P 0497e7bcd8254497bb2635405cde23ee: Bootstrap starting.
I20260807 11:26:34.245095 32438 tablet_bootstrap.cc:654] T 118435a0a85240d684ffb4098d22cfca P 0497e7bcd8254497bb2635405cde23ee: Neither blocks nor log segments found. Creating new log.
I20260807 11:26:34.246541 32440 tablet_bootstrap.cc:492] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5: Bootstrap starting.
I20260807 11:26:34.246606 32438 log.cc:824] T 118435a0a85240d684ffb4098d22cfca P 0497e7bcd8254497bb2635405cde23ee: Log is configured to *not* fsync() on all Append() calls
I20260807 11:26:34.247465 32439 tablet_bootstrap.cc:492] T 118435a0a85240d684ffb4098d22cfca P 6174176c30e34fd1afb4529ac87df331: Bootstrap starting.
I20260807 11:26:34.250195 32440 tablet_bootstrap.cc:654] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5: Neither blocks nor log segments found. Creating new log.
I20260807 11:26:34.251530 32440 log.cc:824] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5: Log is configured to *not* fsync() on all Append() calls
I20260807 11:26:34.251542 32439 tablet_bootstrap.cc:654] T 118435a0a85240d684ffb4098d22cfca P 6174176c30e34fd1afb4529ac87df331: Neither blocks nor log segments found. Creating new log.
I20260807 11:26:34.252794 32439 log.cc:824] T 118435a0a85240d684ffb4098d22cfca P 6174176c30e34fd1afb4529ac87df331: Log is configured to *not* fsync() on all Append() calls
I20260807 11:26:34.253376 32438 tablet_bootstrap.cc:492] T 118435a0a85240d684ffb4098d22cfca P 0497e7bcd8254497bb2635405cde23ee: No bootstrap required, opened a new log
I20260807 11:26:34.253736 32438 ts_tablet_manager.cc:1397] T 118435a0a85240d684ffb4098d22cfca P 0497e7bcd8254497bb2635405cde23ee: Time spent bootstrapping tablet: real 0.012s user 0.011s sys 0.000s
I20260807 11:26:34.254573 32440 tablet_bootstrap.cc:492] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5: No bootstrap required, opened a new log
I20260807 11:26:34.254928 32440 ts_tablet_manager.cc:1397] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 11:26:34.257172 32439 tablet_bootstrap.cc:492] T 118435a0a85240d684ffb4098d22cfca P 6174176c30e34fd1afb4529ac87df331: No bootstrap required, opened a new log
I20260807 11:26:34.257445 32439 ts_tablet_manager.cc:1397] T 118435a0a85240d684ffb4098d22cfca P 6174176c30e34fd1afb4529ac87df331: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 11:26:34.262784 32440 raft_consensus.cc:348] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "152888668c1a4d43bdc59dedb3facdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45307 } } peers { permanent_uuid: "0497e7bcd8254497bb2635405cde23ee" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45945 } } peers { permanent_uuid: "6174176c30e34fd1afb4529ac87df331" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35973 } }
I20260807 11:26:34.263288 32440 raft_consensus.cc:374] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:26:34.263414 32440 raft_consensus.cc:729] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 152888668c1a4d43bdc59dedb3facdd5, State: Initialized, Role: FOLLOWER
I20260807 11:26:34.264138 32440 consensus_queue.cc:260] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "152888668c1a4d43bdc59dedb3facdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45307 } } peers { permanent_uuid: "0497e7bcd8254497bb2635405cde23ee" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45945 } } peers { permanent_uuid: "6174176c30e34fd1afb4529ac87df331" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35973 } }
I20260807 11:26:34.264869 32438 raft_consensus.cc:348] T 118435a0a85240d684ffb4098d22cfca P 0497e7bcd8254497bb2635405cde23ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "152888668c1a4d43bdc59dedb3facdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45307 } } peers { permanent_uuid: "0497e7bcd8254497bb2635405cde23ee" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45945 } } peers { permanent_uuid: "6174176c30e34fd1afb4529ac87df331" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35973 } }
I20260807 11:26:34.265476 32438 raft_consensus.cc:374] T 118435a0a85240d684ffb4098d22cfca P 0497e7bcd8254497bb2635405cde23ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:26:34.265610 32438 raft_consensus.cc:729] T 118435a0a85240d684ffb4098d22cfca P 0497e7bcd8254497bb2635405cde23ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0497e7bcd8254497bb2635405cde23ee, State: Initialized, Role: FOLLOWER
I20260807 11:26:34.266276 32438 consensus_queue.cc:260] T 118435a0a85240d684ffb4098d22cfca P 0497e7bcd8254497bb2635405cde23ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "152888668c1a4d43bdc59dedb3facdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45307 } } peers { permanent_uuid: "0497e7bcd8254497bb2635405cde23ee" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45945 } } peers { permanent_uuid: "6174176c30e34fd1afb4529ac87df331" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35973 } }
I20260807 11:26:34.268155 32439 raft_consensus.cc:348] T 118435a0a85240d684ffb4098d22cfca P 6174176c30e34fd1afb4529ac87df331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "152888668c1a4d43bdc59dedb3facdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45307 } } peers { permanent_uuid: "0497e7bcd8254497bb2635405cde23ee" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45945 } } peers { permanent_uuid: "6174176c30e34fd1afb4529ac87df331" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35973 } }
I20260807 11:26:34.268602 32439 raft_consensus.cc:374] T 118435a0a85240d684ffb4098d22cfca P 6174176c30e34fd1afb4529ac87df331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:26:34.268709 32439 raft_consensus.cc:729] T 118435a0a85240d684ffb4098d22cfca P 6174176c30e34fd1afb4529ac87df331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6174176c30e34fd1afb4529ac87df331, State: Initialized, Role: FOLLOWER
I20260807 11:26:34.269311 32439 consensus_queue.cc:260] T 118435a0a85240d684ffb4098d22cfca P 6174176c30e34fd1afb4529ac87df331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "152888668c1a4d43bdc59dedb3facdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45307 } } peers { permanent_uuid: "0497e7bcd8254497bb2635405cde23ee" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45945 } } peers { permanent_uuid: "6174176c30e34fd1afb4529ac87df331" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35973 } }
I20260807 11:26:34.269909 32438 ts_tablet_manager.cc:1428] T 118435a0a85240d684ffb4098d22cfca P 0497e7bcd8254497bb2635405cde23ee: Time spent starting tablet: real 0.016s user 0.013s sys 0.001s
I20260807 11:26:34.271126 32277 heartbeater.cc:503] Master 127.16.197.62:42873 was elected leader, sending a full tablet report...
I20260807 11:26:34.271644 32418 heartbeater.cc:503] Master 127.16.197.62:42873 was elected leader, sending a full tablet report...
I20260807 11:26:34.271905 32439 ts_tablet_manager.cc:1428] T 118435a0a85240d684ffb4098d22cfca P 6174176c30e34fd1afb4529ac87df331: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260807 11:26:34.273285 32440 ts_tablet_manager.cc:1428] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5: Time spent starting tablet: real 0.018s user 0.006s sys 0.009s
I20260807 11:26:34.290414 32444 raft_consensus.cc:482] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:26:34.290750 32444 raft_consensus.cc:504] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "152888668c1a4d43bdc59dedb3facdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45307 } } peers { permanent_uuid: "0497e7bcd8254497bb2635405cde23ee" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45945 } } peers { permanent_uuid: "6174176c30e34fd1afb4529ac87df331" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35973 } }
I20260807 11:26:34.292563 32444 leader_election.cc:288] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0497e7bcd8254497bb2635405cde23ee (127.16.197.1:45945), 6174176c30e34fd1afb4529ac87df331 (127.16.197.2:35973)
I20260807 11:26:34.299108 32421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:34.293067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:35973 (local address 127.16.197.3:43717)
0807 11:26:34.293456 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:34.293475 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:34.293606 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:34.294229 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:34.294233 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:34.294269 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:26:34.294486 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:34.294504 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.295756 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.295764 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:34.297143 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:34.297156 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.298292 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.298298 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:34.298490 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:34.298514 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 11:26:34.298624 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 11:26:34.298763 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":1}
I20260807 11:26:34.299100 32437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:34.293083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:43717 (local address 127.16.197.2:35973)
0807 11:26:34.293550 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:34.293556 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:34.293576 (+ 20us) server_negotiation.cc:407] Connection header received
0807 11:26:34.293815 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:34.293822 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:34.293945 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:34.294041 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:26:34.294635 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.295607 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.297324 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.298130 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.298307 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:34.298402 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:34.298652 (+ 250us) server_negotiation.cc:299] Negotiation successful
0807 11:26:34.298797 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304}
I20260807 11:26:34.299830 32435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:34.293068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:44831 (local address 127.16.197.1:45945)
0807 11:26:34.293323 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:34.293330 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:34.294197 (+ 867us) server_negotiation.cc:407] Connection header received
0807 11:26:34.294509 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:34.294514 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:34.294620 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:34.294740 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:26:34.295398 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.296102 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.297886 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.298701 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.298898 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:34.299019 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:34.299423 (+ 404us) server_negotiation.cc:299] Negotiation successful
0807 11:26:34.299554 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260807 11:26:34.299860 32449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:34.293105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:45945 (local address 127.16.197.3:44831)
0807 11:26:34.294081 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:34.294099 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:34.294343 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:34.294769 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:34.294773 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:34.294800 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:26:34.295222 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:34.295238 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.296260 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.296265 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:34.297710 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:34.297722 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.298913 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.298919 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:34.299144 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:34.299191 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 11:26:34.299343 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 11:26:34.299492 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":855,"mutex_wait_us":63,"thread_start_us":648,"threads_started":1}
I20260807 11:26:34.300276 32231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "118435a0a85240d684ffb4098d22cfca" candidate_uuid: "152888668c1a4d43bdc59dedb3facdd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6174176c30e34fd1afb4529ac87df331" is_pre_election: true
I20260807 11:26:34.300828 32231 raft_consensus.cc:2454] T 118435a0a85240d684ffb4098d22cfca P 6174176c30e34fd1afb4529ac87df331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 152888668c1a4d43bdc59dedb3facdd5 in term 0.
I20260807 11:26:34.300815 32089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "118435a0a85240d684ffb4098d22cfca" candidate_uuid: "152888668c1a4d43bdc59dedb3facdd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0497e7bcd8254497bb2635405cde23ee" is_pre_election: true
I20260807 11:26:34.301390 32089 raft_consensus.cc:2454] T 118435a0a85240d684ffb4098d22cfca P 0497e7bcd8254497bb2635405cde23ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 152888668c1a4d43bdc59dedb3facdd5 in term 0.
I20260807 11:26:34.301658 32303 leader_election.cc:302] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152888668c1a4d43bdc59dedb3facdd5, 6174176c30e34fd1afb4529ac87df331; no voters:
I20260807 11:26:34.302124 32444 raft_consensus.cc:2781] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:26:34.302377 32444 raft_consensus.cc:482] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:26:34.302510 32444 raft_consensus.cc:3037] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5 [term 0 FOLLOWER]: Advancing to term 1
W20260807 11:26:34.303290 32137 tablet.cc:2369] T 118435a0a85240d684ffb4098d22cfca P 0497e7bcd8254497bb2635405cde23ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:26:34.304889 32444 raft_consensus.cc:504] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "152888668c1a4d43bdc59dedb3facdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45307 } } peers { permanent_uuid: "0497e7bcd8254497bb2635405cde23ee" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45945 } } peers { permanent_uuid: "6174176c30e34fd1afb4529ac87df331" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35973 } }
I20260807 11:26:34.305630 32444 leader_election.cc:288] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5 [CANDIDATE]: Term 1 election: Requested vote from peers 0497e7bcd8254497bb2635405cde23ee (127.16.197.1:45945), 6174176c30e34fd1afb4529ac87df331 (127.16.197.2:35973)
I20260807 11:26:34.305953 32089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "118435a0a85240d684ffb4098d22cfca" candidate_uuid: "152888668c1a4d43bdc59dedb3facdd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0497e7bcd8254497bb2635405cde23ee"
I20260807 11:26:34.306157 32089 raft_consensus.cc:3037] T 118435a0a85240d684ffb4098d22cfca P 0497e7bcd8254497bb2635405cde23ee [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:26:34.306156 32231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "118435a0a85240d684ffb4098d22cfca" candidate_uuid: "152888668c1a4d43bdc59dedb3facdd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6174176c30e34fd1afb4529ac87df331"
I20260807 11:26:34.306386 32231 raft_consensus.cc:3037] T 118435a0a85240d684ffb4098d22cfca P 6174176c30e34fd1afb4529ac87df331 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:26:34.308530 32089 raft_consensus.cc:2454] T 118435a0a85240d684ffb4098d22cfca P 0497e7bcd8254497bb2635405cde23ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 152888668c1a4d43bdc59dedb3facdd5 in term 1.
I20260807 11:26:34.308799 32231 raft_consensus.cc:2454] T 118435a0a85240d684ffb4098d22cfca P 6174176c30e34fd1afb4529ac87df331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 152888668c1a4d43bdc59dedb3facdd5 in term 1.
I20260807 11:26:34.309010 32304 leader_election.cc:302] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0497e7bcd8254497bb2635405cde23ee, 152888668c1a4d43bdc59dedb3facdd5; no voters:
I20260807 11:26:34.309366 32444 raft_consensus.cc:2781] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:26:34.309906 32444 raft_consensus.cc:686] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5 [term 1 LEADER]: Becoming Leader. State: Replica: 152888668c1a4d43bdc59dedb3facdd5, State: Running, Role: LEADER
I20260807 11:26:34.310415 32444 consensus_queue.cc:237] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "152888668c1a4d43bdc59dedb3facdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45307 } } peers { permanent_uuid: "0497e7bcd8254497bb2635405cde23ee" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45945 } } peers { permanent_uuid: "6174176c30e34fd1afb4529ac87df331" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35973 } }
I20260807 11:26:34.315256 31933 catalog_manager.cc:5495] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 152888668c1a4d43bdc59dedb3facdd5 (127.16.197.3). New cstate: current_term: 1 leader_uuid: "152888668c1a4d43bdc59dedb3facdd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "152888668c1a4d43bdc59dedb3facdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0497e7bcd8254497bb2635405cde23ee" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6174176c30e34fd1afb4529ac87df331" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35973 } health_report { overall_health: UNKNOWN } } }
W20260807 11:26:34.332026 32279 tablet.cc:2369] T 118435a0a85240d684ffb4098d22cfca P 6174176c30e34fd1afb4529ac87df331: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:26:34.351814 32434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:34.346185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51948 (local address 127.16.197.3:45307)
0807 11:26:34.346415 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:34.346423 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:34.346446 (+ 23us) server_negotiation.cc:407] Connection header received
0807 11:26:34.346557 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:34.346561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:34.346648 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:34.346811 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:26:34.347286 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.347992 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.349490 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.349925 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.350061 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:34.350570 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:26:34.351173 (+ 603us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:26:34.351342 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:34.351454 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 11:26:34.351538 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 11:26:34.360308 32089 raft_consensus.cc:1261] T 118435a0a85240d684ffb4098d22cfca P 0497e7bcd8254497bb2635405cde23ee [term 1 FOLLOWER]: Refusing update from remote peer 152888668c1a4d43bdc59dedb3facdd5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:26:34.360308 32231 raft_consensus.cc:1261] T 118435a0a85240d684ffb4098d22cfca P 6174176c30e34fd1afb4529ac87df331 [term 1 FOLLOWER]: Refusing update from remote peer 152888668c1a4d43bdc59dedb3facdd5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:26:34.361418 32450 consensus_queue.cc:1035] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0497e7bcd8254497bb2635405cde23ee" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:26:34.361889 32444 consensus_queue.cc:1035] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6174176c30e34fd1afb4529ac87df331" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 11:26:34.372999 32419 tablet.cc:2369] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:26:34.374639 32453 mvcc.cc:205] Tried to move back new op lower bound from 7315873768888614912 to 7315873768699727872. Current Snapshot: MvccSnapshot[applied={T|T < 7315873768888614912}]
I20260807 11:26:34.375995 32454 mvcc.cc:205] Tried to move back new op lower bound from 7315873768888614912 to 7315873768699727872. Current Snapshot: MvccSnapshot[applied={T|T < 7315873768888614912}]
I20260807 11:26:34.388943 31933 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:26:34.389302 31933 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:26:34.393961 31933 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675: Sending DeleteTablet for 3 replicas of tablet 118435a0a85240d684ffb4098d22cfca
I20260807 11:26:34.395394 32350 tablet_service.cc:1514] Processing DeleteTablet for tablet 118435a0a85240d684ffb4098d22cfca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:26:34 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:51942
I20260807 11:26:34.395390 32067 tablet_service.cc:1514] Processing DeleteTablet for tablet 118435a0a85240d684ffb4098d22cfca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:26:34 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:52532
I20260807 11:26:34.396463 32210 tablet_service.cc:1514] Processing DeleteTablet for tablet 118435a0a85240d684ffb4098d22cfca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:26:34 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:50764
I20260807 11:26:34.397403 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 32001
I20260807 11:26:34.408035 32462 tablet_replica.cc:330] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5: stopping tablet replica
I20260807 11:26:34.408150 32464 tablet_replica.cc:330] T 118435a0a85240d684ffb4098d22cfca P 6174176c30e34fd1afb4529ac87df331: stopping tablet replica
I20260807 11:26:34.408658 32462 raft_consensus.cc:2229] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5 [term 1 LEADER]: Raft consensus shutting down.
I20260807 11:26:34.408748 32464 raft_consensus.cc:2229] T 118435a0a85240d684ffb4098d22cfca P 6174176c30e34fd1afb4529ac87df331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:26:34.409487 32464 raft_consensus.cc:2258] T 118435a0a85240d684ffb4098d22cfca P 6174176c30e34fd1afb4529ac87df331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:26:34.409513 32462 raft_consensus.cc:2258] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:26:34.410408 32463 tablet_replica.cc:330] T 118435a0a85240d684ffb4098d22cfca P 0497e7bcd8254497bb2635405cde23ee: stopping tablet replica
I20260807 11:26:34.412817 32462 ts_tablet_manager.cc:1905] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:26:34.412827 32464 ts_tablet_manager.cc:1905] T 118435a0a85240d684ffb4098d22cfca P 6174176c30e34fd1afb4529ac87df331: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 11:26:34 dist-test-slave-2w3w krb5kdc[31887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.1@KRBTEST.COM
I20260807 11:26:34.410871 32463 raft_consensus.cc:2229] T 118435a0a85240d684ffb4098d22cfca P 0497e7bcd8254497bb2635405cde23ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:26:34.419901 32463 raft_consensus.cc:2258] T 118435a0a85240d684ffb4098d22cfca P 0497e7bcd8254497bb2635405cde23ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:26:34.422169 32463 ts_tablet_manager.cc:1905] T 118435a0a85240d684ffb4098d22cfca P 0497e7bcd8254497bb2635405cde23ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:26:34.422205 32462 ts_tablet_manager.cc:1918] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:26:34.422436 32462 log.cc:1196] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/wal/wals/118435a0a85240d684ffb4098d22cfca
I20260807 11:26:34.423111 32462 ts_tablet_manager.cc:1939] T 118435a0a85240d684ffb4098d22cfca P 152888668c1a4d43bdc59dedb3facdd5: Deleting consensus metadata
I20260807 11:26:34.425071 31917 catalog_manager.cc:4841] TS 152888668c1a4d43bdc59dedb3facdd5 (127.16.197.3:45307): tablet 118435a0a85240d684ffb4098d22cfca (table test-table [id=77797c439fff4e2a9a670791f488007b]) successfully deleted
I20260807 11:26:34.428994 32464 ts_tablet_manager.cc:1918] T 118435a0a85240d684ffb4098d22cfca P 6174176c30e34fd1afb4529ac87df331: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:26:34.429258 32464 log.cc:1196] T 118435a0a85240d684ffb4098d22cfca P 6174176c30e34fd1afb4529ac87df331: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/wal/wals/118435a0a85240d684ffb4098d22cfca
I20260807 11:26:34.429888 32464 ts_tablet_manager.cc:1939] T 118435a0a85240d684ffb4098d22cfca P 6174176c30e34fd1afb4529ac87df331: Deleting consensus metadata
I20260807 11:26:34.431877 31918 catalog_manager.cc:4841] TS 6174176c30e34fd1afb4529ac87df331 (127.16.197.2:35973): tablet 118435a0a85240d684ffb4098d22cfca (table test-table [id=77797c439fff4e2a9a670791f488007b]) successfully deleted
I20260807 11:26:34.434798 32463 ts_tablet_manager.cc:1918] T 118435a0a85240d684ffb4098d22cfca P 0497e7bcd8254497bb2635405cde23ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:26:34.435096 32463 log.cc:1196] T 118435a0a85240d684ffb4098d22cfca P 0497e7bcd8254497bb2635405cde23ee: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/wal/wals/118435a0a85240d684ffb4098d22cfca
I20260807 11:26:34.435554 32435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:34.399093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52540 (local address 127.16.197.1:45945)
0807 11:26:34.399341 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:34.399351 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:34.405825 (+ 6474us) server_negotiation.cc:407] Connection header received
0807 11:26:34.407526 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:34.407534 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:34.407649 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:34.407758 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:34.409751 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.411123 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.413098 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.413734 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.413856 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:34.427200 (+ 13344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:34.427215 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:34.427235 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:34.427300 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:34.431111 (+ 3811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:34.432153 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:34.432163 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:34.432167 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:34.432336 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:34.432806 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:34.432811 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:34.432815 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:34.433279 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:34.433598 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:34.434862 (+ 1264us) server_negotiation.cc:299] Negotiation successful
0807 11:26:34.435127 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 11:26:34.435781 32463 ts_tablet_manager.cc:1939] T 118435a0a85240d684ffb4098d22cfca P 0497e7bcd8254497bb2635405cde23ee: Deleting consensus metadata
I20260807 11:26:34.436453 32131 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:26:34.545522 31919 catalog_manager.cc:4841] TS 0497e7bcd8254497bb2635405cde23ee (127.16.197.1:45945): tablet 118435a0a85240d684ffb4098d22cfca (table test-table [id=77797c439fff4e2a9a670791f488007b]) successfully deleted
I20260807 11:26:34.545629 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 32001
I20260807 11:26:34.566674 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 32143
Aug 07 11:26:34 dist-test-slave-2w3w krb5kdc[31887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.2@KRBTEST.COM
I20260807 11:26:34.588548 32437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:34.567636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50772 (local address 127.16.197.2:35973)
0807 11:26:34.567821 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:34.567829 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:34.567864 (+ 35us) server_negotiation.cc:407] Connection header received
0807 11:26:34.568097 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:34.568103 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:34.568186 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:34.568301 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:34.569707 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.570576 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.571983 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.572400 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.572543 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:34.581808 (+ 9265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:34.581822 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:34.581861 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:34.581902 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:34.585236 (+ 3334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:34.586200 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:34.586208 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:34.586210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:34.586339 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:34.586759 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:34.586765 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:34.586768 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:34.587258 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:34.587563 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:34.587912 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 11:26:34.588153 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 11:26:34.589278 32272 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:26:34.740119 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 32143
I20260807 11:26:34.760114 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 32284
Aug 07 11:26:34 dist-test-slave-2w3w krb5kdc[31887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.3@KRBTEST.COM
I20260807 11:26:34.777422 32434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:34.760931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51950 (local address 127.16.197.3:45307)
0807 11:26:34.761132 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:34.761138 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:34.761284 (+ 146us) server_negotiation.cc:407] Connection header received
0807 11:26:34.761508 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:34.761513 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:34.761619 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:34.761736 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:34.763213 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.764049 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.765459 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:34.765928 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:34.766076 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:34.770778 (+ 4702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:34.770796 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:34.770811 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:34.770847 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:34.774281 (+ 3434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:34.775050 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:34.775059 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:34.775061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:34.775228 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:34.775662 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:34.775666 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:34.775668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:34.776044 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:34.776369 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:34.776743 (+ 374us) server_negotiation.cc:299] Negotiation successful
0807 11:26:34.776989 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 11:26:34.778312 32413 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:26:34.905936 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 32284
I20260807 11:26:34.927846 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 31900
I20260807 11:26:34.929090 31967 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:26:35.040827 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 31900
I20260807 11:26:35.067428 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:42873
--webserver_interface=127.16.197.62
--webserver_port=39769
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:36135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:42873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:26:35.407401 32470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:26:35.407728 32470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:26:35.407928 32470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:26:35.415268 32470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:26:35.415380 32470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:26:35.415493 32470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:26:35.415601 32470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:26:35.415688 32470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:26:35.426329 32470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:36135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:42873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:42873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=39769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:26:35.427742 32470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:26:35.429112 32470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:26:35.453859 32477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:35.453819 32479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:35.453837 32476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:35.454828 32470 server_base.cc:1034] running on GCE node
I20260807 11:26:35.455765 32470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:26:35.457433 32470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:26:35.458684 32470 hybrid_clock.cc:648] HybridClock initialized: now 1786101995458623 us; error 70 us; skew 500 ppm
I20260807 11:26:35.459170 32470 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:26:35 dist-test-slave-2w3w krb5kdc[31887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101995, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:26:35.463014 32470 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:26:35.464784 32470 webserver.cc:460] Webserver started at http://127.16.197.62:39769/ using document root <none> and password file <none>
I20260807 11:26:35.465601 32470 fs_manager.cc:359] Metadata directory not provided
I20260807 11:26:35.465786 32470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:26:35.472056 32470 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 11:26:35.475210 32486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:35.476621 32470 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:26:35.476831 32470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "ad8a83ed1c634f0eaf02452b9216f675"
format_stamp: "Formatted at 2026-08-07 11:26:32 on dist-test-slave-2w3w"
server_key: "b9b9d9f56e145c686e6c5d31a63d41ff"
server_key_iv: "31f960df17f3be57b24f0fc8cb28bdc1"
server_key_version: "encryptionkey@0"
I20260807 11:26:35.477963 32470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:26:35.505605 32470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:26:35.509892 32470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:26:35.510191 32470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:26:35.530449 32470 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:42873
I20260807 11:26:35.530491 32540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:42873 every 8 connection(s)
I20260807 11:26:35.532112 32470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:26:35.534289 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 32470
I20260807 11:26:35.535490 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:45945
--local_ip_for_outbound_sockets=127.16.197.1
--tserver_master_addrs=127.16.197.62:42873
--webserver_port=35905
--webserver_interface=127.16.197.1
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:36135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 11:26:35.543784 32541 sys_catalog.cc:263] Verifying existing consensus state
I20260807 11:26:35.548976 32541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675: Bootstrap starting.
I20260807 11:26:35.571934 32541 log.cc:824] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675: Log is configured to *not* fsync() on all Append() calls
I20260807 11:26:35.584846 32541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 11:26:35.585440 32541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675: Bootstrap complete.
I20260807 11:26:35.595391 32541 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8a83ed1c634f0eaf02452b9216f675" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42873 } }
I20260807 11:26:35.596307 32541 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad8a83ed1c634f0eaf02452b9216f675, State: Initialized, Role: FOLLOWER
I20260807 11:26:35.596956 32541 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8a83ed1c634f0eaf02452b9216f675" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42873 } }
I20260807 11:26:35.597234 32541 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:26:35.597424 32541 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:26:35.597615 32541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [term 1 FOLLOWER]: Advancing to term 2
I20260807 11:26:35.603878 32541 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8a83ed1c634f0eaf02452b9216f675" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42873 } }
I20260807 11:26:35.604729 32541 leader_election.cc:302] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad8a83ed1c634f0eaf02452b9216f675; no voters:
I20260807 11:26:35.605873 32541 leader_election.cc:288] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 11:26:35.606149 32547 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 11:26:35.609311 32541 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:26:35.614075 32547 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [term 2 LEADER]: Becoming Leader. State: Replica: ad8a83ed1c634f0eaf02452b9216f675, State: Running, Role: LEADER
I20260807 11:26:35.614862 32547 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8a83ed1c634f0eaf02452b9216f675" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42873 } }
I20260807 11:26:35.618160 32548 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ad8a83ed1c634f0eaf02452b9216f675" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8a83ed1c634f0eaf02452b9216f675" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42873 } } }
I20260807 11:26:35.619056 32548 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [sys.catalog]: This master's current role is: LEADER
I20260807 11:26:35.627050 32549 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad8a83ed1c634f0eaf02452b9216f675. Latest consensus state: current_term: 2 leader_uuid: "ad8a83ed1c634f0eaf02452b9216f675" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8a83ed1c634f0eaf02452b9216f675" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42873 } } }
I20260807 11:26:35.627429 32549 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [sys.catalog]: This master's current role is: LEADER
I20260807 11:26:35.628094 32554 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:26:35.637060 32554 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:26:35.638897 32554 catalog_manager.cc:1259] Loaded cluster ID: 3d5eb72df9f2420db50028e09f26da9c
I20260807 11:26:35.639027 32554 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:26:35.643026 32554 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 11:26:35.669456 32554 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 11:26:35.693384 32554 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:26:35.695840 32554 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675: Loaded TSK: 0
I20260807 11:26:35.698652 32554 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 11:26:35.929412 32544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:26:35.929705 32544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:26:35.929870 32544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:26:35.937551 32544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:26:35.937669 32544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:26:35.937858 32544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:26:35.947911 32544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:36135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:45945
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=35905
--tserver_master_addrs=127.16.197.62:42873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:26:35.949327 32544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:26:35.950711 32544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:26:35.964905 32572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:35.964927 32573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:35.966156 32575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:35.972810 32544 server_base.cc:1034] running on GCE node
I20260807 11:26:35.973515 32544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:26:35.974619 32544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:26:35.975862 32544 hybrid_clock.cc:648] HybridClock initialized: now 1786101995975786 us; error 64 us; skew 500 ppm
I20260807 11:26:35.976199 32544 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:26:35 dist-test-slave-2w3w krb5kdc[31887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101995, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:26:35.979665 32544 init.cc:377] Logged in from keytab as kudu/127.16.197.1@KRBTEST.COM (short username kudu)
I20260807 11:26:35.981316 32544 webserver.cc:460] Webserver started at http://127.16.197.1:35905/ using document root <none> and password file <none>
I20260807 11:26:35.982144 32544 fs_manager.cc:359] Metadata directory not provided
I20260807 11:26:35.982304 32544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:26:35.988286 32544 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 11:26:35.991360 32584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:35.992856 32544 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 11:26:35.993044 32544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "0497e7bcd8254497bb2635405cde23ee"
format_stamp: "Formatted at 2026-08-07 11:26:32 on dist-test-slave-2w3w"
server_key: "21a65431e8ba46ba810568ddc6342b46"
server_key_iv: "f5a48fc07a961cc0679ce8cb02d33032"
server_key_version: "encryptionkey@0"
I20260807 11:26:35.994076 32544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:26:36.032920 32544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:26:36.037480 32544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:26:36.037765 32544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:26:36.038793 32544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:26:36.040254 32544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:26:36.040380 32544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:36.040517 32544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:26:36.040604 32544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:36.075913 32544 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:45945
I20260807 11:26:36.076011 32701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:45945 every 8 connection(s)
I20260807 11:26:36.077508 32544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:26:36.087203 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 32544
I20260807 11:26:36.087913 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:35973
--local_ip_for_outbound_sockets=127.16.197.2
--tserver_master_addrs=127.16.197.62:42873
--webserver_port=38967
--webserver_interface=127.16.197.2
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:36135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 11:26:36 dist-test-slave-2w3w krb5kdc[31887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101995, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:26:36.103698 32705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:36.080913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.1:45605)
0807 11:26:36.081753 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:36.081801 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:36.083709 (+ 1908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:36.085035 (+ 1326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:36.085042 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:36.085576 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:26:36.086128 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:36.086146 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:36.087917 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:36.087923 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:36.090287 (+ 2364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:36.090309 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:36.091428 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:36.091437 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:36.091792 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:36.092564 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:26:36.092602 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:26:36.095403 (+ 2801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:26:36.099185 (+ 3782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:36.099195 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:36.099210 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:36.099646 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:36.100115 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:36.100120 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:36.100124 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:36.100307 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:36.101060 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:26:36.101069 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:26:36.101406 (+ 337us) client_negotiation.cc:769] Sending connection context
0807 11:26:36.101685 (+ 279us) client_negotiation.cc:240] Negotiation successful
0807 11:26:36.102278 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":329,"threads_started":1}
I20260807 11:26:36.103698 32704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:36.080608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:45605 (local address 127.16.197.62:42873)
0807 11:26:36.081248 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:36.081270 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:36.081930 (+ 660us) server_negotiation.cc:407] Connection header received
0807 11:26:36.084219 (+ 2289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:36.084232 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:36.084559 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:36.084856 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:36.086340 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:36.087732 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:36.090460 (+ 2728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:36.091175 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:36.091312 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:36.095689 (+ 4377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:36.095697 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:36.095710 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:36.095755 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:36.099040 (+ 3285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:36.099794 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:36.099803 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:36.099810 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:36.099990 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:36.100422 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:36.100428 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:36.100432 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:36.100908 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:36.101119 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:36.101915 (+ 796us) server_negotiation.cc:299] Negotiation successful
0807 11:26:36.102311 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":245,"threads_started":1}
I20260807 11:26:36.106890 32702 heartbeater.cc:347] Connected to a master server at 127.16.197.62:42873
I20260807 11:26:36.107273 32702 heartbeater.cc:464] Registering TS with master...
I20260807 11:26:36.108210 32702 heartbeater.cc:511] Master 127.16.197.62:42873 requested a full tablet report, sending...
I20260807 11:26:36.111019 32504 ts_manager.cc:194] Registered new tserver with Master: 0497e7bcd8254497bb2635405cde23ee (127.16.197.1:45945)
I20260807 11:26:36.113425 32504 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.1@KRBTEST.COM'} at 127.16.197.1:45605
W20260807 11:26:36.438683 32706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:26:36.439047 32706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:26:36.439162 32706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:26:36.446626 32706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:26:36.446734 32706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:26:36.446861 32706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:26:36.456864 32706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:36135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:35973
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=38967
--tserver_master_addrs=127.16.197.62:42873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:26:36.458230 32706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:26:36.459554 32706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:26:36.473030 32711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:36.472955 32712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:36.472922 32714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:36.473863 32706 server_base.cc:1034] running on GCE node
I20260807 11:26:36.474529 32706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:26:36.475566 32706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:26:36.476775 32706 hybrid_clock.cc:648] HybridClock initialized: now 1786101996476727 us; error 44 us; skew 500 ppm
I20260807 11:26:36.477118 32706 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:26:36 dist-test-slave-2w3w krb5kdc[31887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101996, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:26:36.480230 32706 init.cc:377] Logged in from keytab as kudu/127.16.197.2@KRBTEST.COM (short username kudu)
I20260807 11:26:36.481693 32706 webserver.cc:460] Webserver started at http://127.16.197.2:38967/ using document root <none> and password file <none>
I20260807 11:26:36.482460 32706 fs_manager.cc:359] Metadata directory not provided
I20260807 11:26:36.482612 32706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:26:36.488189 32706 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 11:26:36.491093 32721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:36.492475 32706 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 11:26:36.492660 32706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "6174176c30e34fd1afb4529ac87df331"
format_stamp: "Formatted at 2026-08-07 11:26:33 on dist-test-slave-2w3w"
server_key: "29c20b461afe09c2275938a988d709c9"
server_key_iv: "b9a21728bf26fbf0bfa0d98598a5bc97"
server_key_version: "encryptionkey@0"
I20260807 11:26:36.493671 32706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:26:36.507453 32706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:26:36.511548 32706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:26:36.511832 32706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:26:36.512822 32706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:26:36.514245 32706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:26:36.514386 32706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:36.514515 32706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:26:36.514604 32706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:36.549850 32706 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:35973
I20260807 11:26:36.549902 373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:35973 every 8 connection(s)
I20260807 11:26:36.551441 32706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:26:36.558606 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 32706
I20260807 11:26:36.559398 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:45307
--local_ip_for_outbound_sockets=127.16.197.3
--tserver_master_addrs=127.16.197.62:42873
--webserver_port=36869
--webserver_interface=127.16.197.3
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:36135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 11:26:36 dist-test-slave-2w3w krb5kdc[31887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101996, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:26:36.574156 32704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:36.553910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:36743 (local address 127.16.197.62:42873)
0807 11:26:36.554078 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:36.554083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:36.555220 (+ 1137us) server_negotiation.cc:407] Connection header received
0807 11:26:36.557033 (+ 1813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:36.557038 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:36.557138 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:36.557251 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:36.559101 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:36.560116 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:36.562812 (+ 2696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:36.563319 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:36.563435 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:36.568290 (+ 4855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:36.568305 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:36.568308 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:36.568344 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:36.571080 (+ 2736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:36.571834 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:36.571840 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:36.571843 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:36.571982 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:36.572426 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:36.572430 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:36.572432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:36.572692 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:36.572846 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:36.573551 (+ 705us) server_negotiation.cc:299] Negotiation successful
0807 11:26:36.573801 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 11:26:36.575261 376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:36.554283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.2:36743)
0807 11:26:36.555065 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:36.555103 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:36.556772 (+ 1669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:36.557496 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:36.557504 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:36.558011 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:26:36.558862 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:36.558889 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:36.560284 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:36.560293 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:36.562585 (+ 2292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:36.562604 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:36.563569 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:36.563578 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:36.563920 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:36.564732 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:26:36.564770 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:26:36.568027 (+ 3257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:26:36.571231 (+ 3204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:36.571241 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:36.571259 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:36.571699 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:36.572097 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:36.572102 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:36.572105 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:36.572305 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:36.572832 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:26:36.572839 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:26:36.573177 (+ 338us) client_negotiation.cc:769] Sending connection context
0807 11:26:36.573445 (+ 268us) client_negotiation.cc:240] Negotiation successful
0807 11:26:36.574004 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":277,"threads_started":1}
I20260807 11:26:36.576885 374 heartbeater.cc:347] Connected to a master server at 127.16.197.62:42873
I20260807 11:26:36.577222 374 heartbeater.cc:464] Registering TS with master...
I20260807 11:26:36.577982 374 heartbeater.cc:511] Master 127.16.197.62:42873 requested a full tablet report, sending...
I20260807 11:26:36.579499 32504 ts_manager.cc:194] Registered new tserver with Master: 6174176c30e34fd1afb4529ac87df331 (127.16.197.2:35973)
I20260807 11:26:36.580343 32504 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.2@KRBTEST.COM'} at 127.16.197.2:36743
W20260807 11:26:36.905238 377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:26:36.905579 377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:26:36.905742 377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:26:36.913273 377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:26:36.913442 377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:26:36.913612 377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:26:36.923480 377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:36135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:45307
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=36869
--tserver_master_addrs=127.16.197.62:42873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:26:36.924930 377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:26:36.926306 377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:26:36.940011 383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:36.940054 382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:36.940011 385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:36.941226 377 server_base.cc:1034] running on GCE node
I20260807 11:26:36.941895 377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:26:36.942833 377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:26:36.944059 377 hybrid_clock.cc:648] HybridClock initialized: now 1786101996944006 us; error 65 us; skew 500 ppm
I20260807 11:26:36.944380 377 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:26:36 dist-test-slave-2w3w krb5kdc[31887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101996, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:26:36.947445 377 init.cc:377] Logged in from keytab as kudu/127.16.197.3@KRBTEST.COM (short username kudu)
I20260807 11:26:36.948982 377 webserver.cc:460] Webserver started at http://127.16.197.3:36869/ using document root <none> and password file <none>
I20260807 11:26:36.949648 377 fs_manager.cc:359] Metadata directory not provided
I20260807 11:26:36.949798 377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:26:36.955204 377 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 11:26:36.957782 392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:36.959159 377 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:26:36.959326 377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "152888668c1a4d43bdc59dedb3facdd5"
format_stamp: "Formatted at 2026-08-07 11:26:34 on dist-test-slave-2w3w"
server_key: "b9c421176e38932e8ddd7827d62b9be9"
server_key_iv: "712fb01d6d4a437b94f68ce11ee62167"
server_key_version: "encryptionkey@0"
I20260807 11:26:36.960325 377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:26:36.978267 377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:26:36.982671 377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:26:36.983053 377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:26:36.984114 377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:26:36.985502 377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:26:36.985626 377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:36.985761 377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:26:36.985848 377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:37.020952 377 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:45307
I20260807 11:26:37.021009 514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:45307 every 8 connection(s)
I20260807 11:26:37.022433 377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:26:37.030838 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 377
Aug 07 11:26:37 dist-test-slave-2w3w krb5kdc[31887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101996, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:26:37.045190 32704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:37.025150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:37127 (local address 127.16.197.62:42873)
0807 11:26:37.025331 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:37.025336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:37.026362 (+ 1026us) server_negotiation.cc:407] Connection header received
0807 11:26:37.028256 (+ 1894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:37.028261 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:37.028353 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:37.028463 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:37.029949 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.030813 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.032765 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.033435 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.033586 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:37.037616 (+ 4030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:37.037621 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:37.037624 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:37.037661 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:37.041228 (+ 3567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:37.042213 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:37.042217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:37.042220 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:37.042360 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:37.042848 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:37.042855 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:37.042858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:37.043264 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:37.043431 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:37.044218 (+ 787us) server_negotiation.cc:299] Negotiation successful
0807 11:26:37.044623 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 11:26:37.046100 517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:37.025483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.3:37127)
0807 11:26:37.026179 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:37.026231 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:37.027988 (+ 1757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:37.028709 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:37.028720 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:37.029223 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:26:37.029779 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:37.029796 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.030996 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.031002 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:37.032586 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:37.032600 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.033659 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.033664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:37.033937 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:37.034644 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:26:37.034694 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:26:37.037368 (+ 2674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:26:37.041391 (+ 4023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:37.041398 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:37.041411 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:37.042081 (+ 670us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:37.042474 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:37.042481 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:37.042488 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:37.042713 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:37.043420 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:26:37.043429 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:26:37.043831 (+ 402us) client_negotiation.cc:769] Sending connection context
0807 11:26:37.044065 (+ 234us) client_negotiation.cc:240] Negotiation successful
0807 11:26:37.044530 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":285,"threads_started":1}
I20260807 11:26:37.049007 515 heartbeater.cc:347] Connected to a master server at 127.16.197.62:42873
I20260807 11:26:37.049088 518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:37.031780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38862 (local address 127.16.197.62:42873)
0807 11:26:37.032280 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:37.032285 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:37.032679 (+ 394us) server_negotiation.cc:407] Connection header received
0807 11:26:37.032878 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:37.032882 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:37.032940 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:37.033020 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:37.034771 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.035991 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.037521 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.038022 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.038114 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:37.041633 (+ 3519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:37.041641 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:37.041647 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:37.041684 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:37.045054 (+ 3370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:37.045917 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:37.045924 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:37.045926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:37.046107 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:37.047022 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:37.047029 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:37.047034 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:37.047481 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:37.047769 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:37.048413 (+ 644us) server_negotiation.cc:299] Negotiation successful
0807 11:26:37.048670 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":263,"threads_started":1}
I20260807 11:26:37.049348 515 heartbeater.cc:464] Registering TS with master...
I20260807 11:26:37.050298 515 heartbeater.cc:511] Master 127.16.197.62:42873 requested a full tablet report, sending...
I20260807 11:26:37.052074 32504 ts_manager.cc:194] Registered new tserver with Master: 152888668c1a4d43bdc59dedb3facdd5 (127.16.197.3:45307)
I20260807 11:26:37.053279 32504 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.3@KRBTEST.COM'} at 127.16.197.3:37127
I20260807 11:26:37.064347 17172 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 11:26:37.071285 518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:37.065285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38868 (local address 127.16.197.62:42873)
0807 11:26:37.065485 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:37.065494 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:37.065942 (+ 448us) server_negotiation.cc:407] Connection header received
0807 11:26:37.066137 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:37.066140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:37.066286 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:37.066414 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:26:37.066850 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.067600 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.069272 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.069677 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.069777 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:37.070196 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:26:37.070675 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:26:37.070779 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:37.070875 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 11:26:37.070947 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 11:26:37.072153 32504 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:38868:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 11:26:37.079530 32504 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:26:37.117659 32702 heartbeater.cc:503] Master 127.16.197.62:42873 was elected leader, sending a full tablet report...
I20260807 11:26:37.117772 528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:37.091449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:35973 (local address 127.0.0.1:50780)
0807 11:26:37.107980 (+ 16531us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:37.108024 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:37.108263 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:37.109407 (+ 1144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:37.109415 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:37.109456 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:26:37.109952 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:37.109969 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.111865 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.111870 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:37.114381 (+ 2511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:37.114397 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.116630 (+ 2233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.116638 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:37.116904 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:37.116963 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 11:26:37.117094 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 11:26:37.117285 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16264,"mutex_wait_us":24,"thread_start_us":19289,"threads_started":1}
I20260807 11:26:37.118569 525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:37.091461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50780 (local address 127.16.197.2:35973)
0807 11:26:37.106358 (+ 14897us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:37.106372 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:37.108149 (+ 1777us) server_negotiation.cc:407] Connection header received
0807 11:26:37.108550 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:37.108562 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:37.108866 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:37.109073 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:26:37.110187 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.111723 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.114559 (+ 2836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.116109 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.116258 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:37.116472 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:37.118016 (+ 1544us) server_negotiation.cc:299] Negotiation successful
0807 11:26:37.118219 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14644,"thread_start_us":14451,"threads_started":1}
I20260807 11:26:37.119035 527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:37.091513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:45945 (local address 127.0.0.1:52544)
0807 11:26:37.108969 (+ 17456us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:37.108990 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:37.109099 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:37.110350 (+ 1251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:37.110358 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:37.110397 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:26:37.111088 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:37.111099 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.113714 (+ 2615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.113721 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:37.115459 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:37.115472 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.118199 (+ 2727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.118209 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:37.118404 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:37.118439 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 11:26:37.118524 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 11:26:37.118668 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17305,"thread_start_us":17163,"threads_started":1}
I20260807 11:26:37.119887 529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:37.092277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52544 (local address 127.16.197.1:45945)
0807 11:26:37.106815 (+ 14538us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:37.106827 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:37.109226 (+ 2399us) server_negotiation.cc:407] Connection header received
0807 11:26:37.109658 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:37.109665 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:37.109922 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:37.110153 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:26:37.111306 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.112712 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.115626 (+ 2914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.117163 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.117310 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:37.117537 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:37.119399 (+ 1862us) server_negotiation.cc:299] Negotiation successful
0807 11:26:37.119582 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14311,"thread_start_us":14154,"threads_started":1}
I20260807 11:26:37.122325 304 tablet_service.cc:1467] Processing CreateTablet for tablet 6ad37afce1d6430ea1b03b4f5de92e1d (DEFAULT_TABLE table=test-table [id=aae9ae5023ab4ce68859158fe7148f36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:26:37.122838 32633 tablet_service.cc:1467] Processing CreateTablet for tablet 6ad37afce1d6430ea1b03b4f5de92e1d (DEFAULT_TABLE table=test-table [id=aae9ae5023ab4ce68859158fe7148f36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:26:37.123620 304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ad37afce1d6430ea1b03b4f5de92e1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:26:37.124064 32633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ad37afce1d6430ea1b03b4f5de92e1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:26:37.127665 526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:37.092030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:45307 (local address 127.0.0.1:51962)
0807 11:26:37.111117 (+ 19087us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:37.111137 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:37.111267 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:37.113423 (+ 2156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:37.113426 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:37.113447 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:26:37.113810 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:37.113820 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.115659 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.115664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:37.117760 (+ 2096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:37.117793 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.126772 (+ 8979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.126779 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:37.127003 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:37.127047 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 11:26:37.127147 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 11:26:37.127297 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18958,"mutex_wait_us":3,"thread_start_us":15523,"threads_started":1}
I20260807 11:26:37.127707 530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:37.092768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51962 (local address 127.16.197.3:45307)
0807 11:26:37.109439 (+ 16671us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:37.109447 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:37.112882 (+ 3435us) server_negotiation.cc:407] Connection header received
0807 11:26:37.112988 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:37.112996 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:37.113242 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:37.113430 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:26:37.114019 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.115536 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.118305 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.126207 (+ 7902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.126357 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:37.126601 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:37.127206 (+ 605us) server_negotiation.cc:299] Negotiation successful
0807 11:26:37.127378 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14401,"thread_start_us":14242,"threads_started":1}
I20260807 11:26:37.131708 445 tablet_service.cc:1467] Processing CreateTablet for tablet 6ad37afce1d6430ea1b03b4f5de92e1d (DEFAULT_TABLE table=test-table [id=aae9ae5023ab4ce68859158fe7148f36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:26:37.133205 445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ad37afce1d6430ea1b03b4f5de92e1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:26:37.136742 531 tablet_bootstrap.cc:492] T 6ad37afce1d6430ea1b03b4f5de92e1d P 6174176c30e34fd1afb4529ac87df331: Bootstrap starting.
I20260807 11:26:37.139981 532 tablet_bootstrap.cc:492] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee: Bootstrap starting.
I20260807 11:26:37.142769 531 tablet_bootstrap.cc:654] T 6ad37afce1d6430ea1b03b4f5de92e1d P 6174176c30e34fd1afb4529ac87df331: Neither blocks nor log segments found. Creating new log.
I20260807 11:26:37.143721 532 tablet_bootstrap.cc:654] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee: Neither blocks nor log segments found. Creating new log.
I20260807 11:26:37.144245 531 log.cc:824] T 6ad37afce1d6430ea1b03b4f5de92e1d P 6174176c30e34fd1afb4529ac87df331: Log is configured to *not* fsync() on all Append() calls
I20260807 11:26:37.145038 532 log.cc:824] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee: Log is configured to *not* fsync() on all Append() calls
I20260807 11:26:37.147899 536 tablet_bootstrap.cc:492] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5: Bootstrap starting.
I20260807 11:26:37.148026 531 tablet_bootstrap.cc:492] T 6ad37afce1d6430ea1b03b4f5de92e1d P 6174176c30e34fd1afb4529ac87df331: No bootstrap required, opened a new log
I20260807 11:26:37.148298 531 ts_tablet_manager.cc:1397] T 6ad37afce1d6430ea1b03b4f5de92e1d P 6174176c30e34fd1afb4529ac87df331: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260807 11:26:37.150694 532 tablet_bootstrap.cc:492] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee: No bootstrap required, opened a new log
I20260807 11:26:37.150898 532 ts_tablet_manager.cc:1397] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260807 11:26:37.151698 536 tablet_bootstrap.cc:654] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5: Neither blocks nor log segments found. Creating new log.
I20260807 11:26:37.152945 536 log.cc:824] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5: Log is configured to *not* fsync() on all Append() calls
I20260807 11:26:37.156378 536 tablet_bootstrap.cc:492] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5: No bootstrap required, opened a new log
I20260807 11:26:37.156649 536 ts_tablet_manager.cc:1397] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 11:26:37.158793 532 raft_consensus.cc:348] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6174176c30e34fd1afb4529ac87df331" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35973 } } peers { permanent_uuid: "0497e7bcd8254497bb2635405cde23ee" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45945 } } peers { permanent_uuid: "152888668c1a4d43bdc59dedb3facdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45307 } }
I20260807 11:26:37.159255 532 raft_consensus.cc:374] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:26:37.159369 532 raft_consensus.cc:729] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0497e7bcd8254497bb2635405cde23ee, State: Initialized, Role: FOLLOWER
I20260807 11:26:37.159323 531 raft_consensus.cc:348] T 6ad37afce1d6430ea1b03b4f5de92e1d P 6174176c30e34fd1afb4529ac87df331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6174176c30e34fd1afb4529ac87df331" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35973 } } peers { permanent_uuid: "0497e7bcd8254497bb2635405cde23ee" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45945 } } peers { permanent_uuid: "152888668c1a4d43bdc59dedb3facdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45307 } }
I20260807 11:26:37.159746 531 raft_consensus.cc:374] T 6ad37afce1d6430ea1b03b4f5de92e1d P 6174176c30e34fd1afb4529ac87df331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:26:37.159832 531 raft_consensus.cc:729] T 6ad37afce1d6430ea1b03b4f5de92e1d P 6174176c30e34fd1afb4529ac87df331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6174176c30e34fd1afb4529ac87df331, State: Initialized, Role: FOLLOWER
I20260807 11:26:37.160144 532 consensus_queue.cc:260] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6174176c30e34fd1afb4529ac87df331" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35973 } } peers { permanent_uuid: "0497e7bcd8254497bb2635405cde23ee" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45945 } } peers { permanent_uuid: "152888668c1a4d43bdc59dedb3facdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45307 } }
I20260807 11:26:37.160494 531 consensus_queue.cc:260] T 6ad37afce1d6430ea1b03b4f5de92e1d P 6174176c30e34fd1afb4529ac87df331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6174176c30e34fd1afb4529ac87df331" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35973 } } peers { permanent_uuid: "0497e7bcd8254497bb2635405cde23ee" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45945 } } peers { permanent_uuid: "152888668c1a4d43bdc59dedb3facdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45307 } }
I20260807 11:26:37.162806 374 heartbeater.cc:503] Master 127.16.197.62:42873 was elected leader, sending a full tablet report...
I20260807 11:26:37.164318 532 ts_tablet_manager.cc:1428] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee: Time spent starting tablet: real 0.013s user 0.011s sys 0.000s
I20260807 11:26:37.165691 531 ts_tablet_manager.cc:1428] T 6ad37afce1d6430ea1b03b4f5de92e1d P 6174176c30e34fd1afb4529ac87df331: Time spent starting tablet: real 0.017s user 0.015s sys 0.000s
I20260807 11:26:37.167740 536 raft_consensus.cc:348] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6174176c30e34fd1afb4529ac87df331" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35973 } } peers { permanent_uuid: "0497e7bcd8254497bb2635405cde23ee" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45945 } } peers { permanent_uuid: "152888668c1a4d43bdc59dedb3facdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45307 } }
I20260807 11:26:37.168208 536 raft_consensus.cc:374] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:26:37.168350 536 raft_consensus.cc:729] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 152888668c1a4d43bdc59dedb3facdd5, State: Initialized, Role: FOLLOWER
I20260807 11:26:37.168979 536 consensus_queue.cc:260] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6174176c30e34fd1afb4529ac87df331" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35973 } } peers { permanent_uuid: "0497e7bcd8254497bb2635405cde23ee" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45945 } } peers { permanent_uuid: "152888668c1a4d43bdc59dedb3facdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45307 } }
I20260807 11:26:37.170288 515 heartbeater.cc:503] Master 127.16.197.62:42873 was elected leader, sending a full tablet report...
I20260807 11:26:37.170710 536 ts_tablet_manager.cc:1428] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260807 11:26:37.216358 540 raft_consensus.cc:482] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:26:37.216699 540 raft_consensus.cc:504] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6174176c30e34fd1afb4529ac87df331" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35973 } } peers { permanent_uuid: "0497e7bcd8254497bb2635405cde23ee" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45945 } } peers { permanent_uuid: "152888668c1a4d43bdc59dedb3facdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45307 } }
I20260807 11:26:37.218520 540 leader_election.cc:288] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6174176c30e34fd1afb4529ac87df331 (127.16.197.2:35973), 0497e7bcd8254497bb2635405cde23ee (127.16.197.1:45945)
I20260807 11:26:37.225112 529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:37.219161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:47101 (local address 127.16.197.1:45945)
0807 11:26:37.219373 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:37.219381 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:37.219403 (+ 22us) server_negotiation.cc:407] Connection header received
0807 11:26:37.219635 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:37.219639 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:37.219699 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:37.219804 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:26:37.220259 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.221268 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.223195 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.224290 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.224414 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:37.224468 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:37.224824 (+ 356us) server_negotiation.cc:299] Negotiation successful
0807 11:26:37.224929 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 11:26:37.225270 517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:37.218913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:45945 (local address 127.16.197.3:47101)
0807 11:26:37.219318 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:37.219339 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:37.219475 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:37.219828 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:37.219833 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:37.219861 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:26:37.220100 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:37.220115 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.221414 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.221420 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:37.223035 (+ 1615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:37.223048 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.224453 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.224459 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:37.224656 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:37.224696 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 11:26:37.224805 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 11:26:37.224956 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":11}
I20260807 11:26:37.226257 525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:37.219036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:36359 (local address 127.16.197.2:35973)
0807 11:26:37.219263 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:37.219272 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:37.220033 (+ 761us) server_negotiation.cc:407] Connection header received
0807 11:26:37.220165 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:37.220172 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:37.220258 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:37.220333 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:26:37.220969 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.221963 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.223970 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.225110 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.225296 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:37.225376 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:37.225871 (+ 495us) server_negotiation.cc:299] Negotiation successful
0807 11:26:37.226002 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 11:26:37.226346 541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:37.219004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:35973 (local address 127.16.197.3:36359)
0807 11:26:37.219928 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:37.219942 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:37.220051 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:37.220395 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:37.220399 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:37.220425 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:26:37.220779 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:37.220791 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.222123 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.222127 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:37.223794 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:37.223810 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.225516 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.225522 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:37.225715 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:37.225749 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 11:26:37.225839 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 11:26:37.225982 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":793,"mutex_wait_us":1,"thread_start_us":584,"threads_started":1}
I20260807 11:26:37.226294 32655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ad37afce1d6430ea1b03b4f5de92e1d" candidate_uuid: "152888668c1a4d43bdc59dedb3facdd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0497e7bcd8254497bb2635405cde23ee" is_pre_election: true
I20260807 11:26:37.226894 32655 raft_consensus.cc:2454] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 152888668c1a4d43bdc59dedb3facdd5 in term 0.
I20260807 11:26:37.227315 325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ad37afce1d6430ea1b03b4f5de92e1d" candidate_uuid: "152888668c1a4d43bdc59dedb3facdd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6174176c30e34fd1afb4529ac87df331" is_pre_election: true
I20260807 11:26:37.227684 395 leader_election.cc:302] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0497e7bcd8254497bb2635405cde23ee, 152888668c1a4d43bdc59dedb3facdd5; no voters:
I20260807 11:26:37.227749 325 raft_consensus.cc:2454] T 6ad37afce1d6430ea1b03b4f5de92e1d P 6174176c30e34fd1afb4529ac87df331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 152888668c1a4d43bdc59dedb3facdd5 in term 0.
I20260807 11:26:37.228078 540 raft_consensus.cc:2781] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:26:37.228230 540 raft_consensus.cc:482] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:26:37.228364 540 raft_consensus.cc:3037] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:26:37.230618 540 raft_consensus.cc:504] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6174176c30e34fd1afb4529ac87df331" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35973 } } peers { permanent_uuid: "0497e7bcd8254497bb2635405cde23ee" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45945 } } peers { permanent_uuid: "152888668c1a4d43bdc59dedb3facdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45307 } }
I20260807 11:26:37.231336 540 leader_election.cc:288] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5 [CANDIDATE]: Term 1 election: Requested vote from peers 6174176c30e34fd1afb4529ac87df331 (127.16.197.2:35973), 0497e7bcd8254497bb2635405cde23ee (127.16.197.1:45945)
I20260807 11:26:37.231801 325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ad37afce1d6430ea1b03b4f5de92e1d" candidate_uuid: "152888668c1a4d43bdc59dedb3facdd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6174176c30e34fd1afb4529ac87df331"
I20260807 11:26:37.231878 32655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ad37afce1d6430ea1b03b4f5de92e1d" candidate_uuid: "152888668c1a4d43bdc59dedb3facdd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0497e7bcd8254497bb2635405cde23ee"
I20260807 11:26:37.232035 325 raft_consensus.cc:3037] T 6ad37afce1d6430ea1b03b4f5de92e1d P 6174176c30e34fd1afb4529ac87df331 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:26:37.232075 32655 raft_consensus.cc:3037] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:26:37.235431 325 raft_consensus.cc:2454] T 6ad37afce1d6430ea1b03b4f5de92e1d P 6174176c30e34fd1afb4529ac87df331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 152888668c1a4d43bdc59dedb3facdd5 in term 1.
I20260807 11:26:37.235431 32655 raft_consensus.cc:2454] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 152888668c1a4d43bdc59dedb3facdd5 in term 1.
I20260807 11:26:37.235857 394 leader_election.cc:302] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 152888668c1a4d43bdc59dedb3facdd5, 6174176c30e34fd1afb4529ac87df331; no voters:
I20260807 11:26:37.236153 540 raft_consensus.cc:2781] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:26:37.236603 540 raft_consensus.cc:686] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5 [term 1 LEADER]: Becoming Leader. State: Replica: 152888668c1a4d43bdc59dedb3facdd5, State: Running, Role: LEADER
I20260807 11:26:37.237357 540 consensus_queue.cc:237] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6174176c30e34fd1afb4529ac87df331" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35973 } } peers { permanent_uuid: "0497e7bcd8254497bb2635405cde23ee" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45945 } } peers { permanent_uuid: "152888668c1a4d43bdc59dedb3facdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45307 } }
I20260807 11:26:37.242354 32504 catalog_manager.cc:5495] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 152888668c1a4d43bdc59dedb3facdd5 (127.16.197.3). New cstate: current_term: 1 leader_uuid: "152888668c1a4d43bdc59dedb3facdd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6174176c30e34fd1afb4529ac87df331" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35973 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0497e7bcd8254497bb2635405cde23ee" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "152888668c1a4d43bdc59dedb3facdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45307 } health_report { overall_health: HEALTHY } } }
W20260807 11:26:37.274091 516 tablet.cc:2369] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:26:37.277069 530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:37.271615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51964 (local address 127.16.197.3:45307)
0807 11:26:37.271816 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:37.271821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:37.271839 (+ 18us) server_negotiation.cc:407] Connection header received
0807 11:26:37.272034 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:37.272038 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:37.272115 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:37.272238 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:26:37.272622 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.273315 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.274871 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.275354 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.275462 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:37.275991 (+ 529us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:26:37.276488 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:26:37.276599 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:37.276727 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 11:26:37.276792 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 11:26:37.285321 32655 raft_consensus.cc:1261] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [term 1 FOLLOWER]: Refusing update from remote peer 152888668c1a4d43bdc59dedb3facdd5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:26:37.285341 325 raft_consensus.cc:1261] T 6ad37afce1d6430ea1b03b4f5de92e1d P 6174176c30e34fd1afb4529ac87df331 [term 1 FOLLOWER]: Refusing update from remote peer 152888668c1a4d43bdc59dedb3facdd5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:26:37.286288 542 consensus_queue.cc:1035] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6174176c30e34fd1afb4529ac87df331" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:26:37.286619 540 consensus_queue.cc:1035] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0497e7bcd8254497bb2635405cde23ee" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:26:37.313907 32503 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:38868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:26:37.314203 32503 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:38868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:26:37.317898 32503 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675: Sending DeleteTablet for 3 replicas of tablet 6ad37afce1d6430ea1b03b4f5de92e1d
I20260807 11:26:37.319144 304 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ad37afce1d6430ea1b03b4f5de92e1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:26:37 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:50780
I20260807 11:26:37.319183 32633 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ad37afce1d6430ea1b03b4f5de92e1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:26:37 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:52544
I20260807 11:26:37.319414 445 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ad37afce1d6430ea1b03b4f5de92e1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:26:37 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:51962
I20260807 11:26:37.320549 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 32544
I20260807 11:26:37.331866 557 tablet_replica.cc:330] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5: stopping tablet replica
I20260807 11:26:37.333002 557 raft_consensus.cc:2229] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5 [term 1 LEADER]: Raft consensus shutting down.
I20260807 11:26:37.334396 557 raft_consensus.cc:2258] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:26:37.335481 554 tablet_replica.cc:330] T 6ad37afce1d6430ea1b03b4f5de92e1d P 6174176c30e34fd1afb4529ac87df331: stopping tablet replica
I20260807 11:26:37.336282 554 raft_consensus.cc:2229] T 6ad37afce1d6430ea1b03b4f5de92e1d P 6174176c30e34fd1afb4529ac87df331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:26:37.336926 554 raft_consensus.cc:2258] T 6ad37afce1d6430ea1b03b4f5de92e1d P 6174176c30e34fd1afb4529ac87df331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:26:37.339185 554 ts_tablet_manager.cc:1905] T 6ad37afce1d6430ea1b03b4f5de92e1d P 6174176c30e34fd1afb4529ac87df331: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:26:37.345935 557 ts_tablet_manager.cc:1905] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:26:37.346764 554 ts_tablet_manager.cc:1918] T 6ad37afce1d6430ea1b03b4f5de92e1d P 6174176c30e34fd1afb4529ac87df331: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:26:37.347038 554 log.cc:1196] T 6ad37afce1d6430ea1b03b4f5de92e1d P 6174176c30e34fd1afb4529ac87df331: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/wal/wals/6ad37afce1d6430ea1b03b4f5de92e1d
I20260807 11:26:37.347718 554 ts_tablet_manager.cc:1939] T 6ad37afce1d6430ea1b03b4f5de92e1d P 6174176c30e34fd1afb4529ac87df331: Deleting consensus metadata
I20260807 11:26:37.349728 32488 catalog_manager.cc:4841] TS 6174176c30e34fd1afb4529ac87df331 (127.16.197.2:35973): tablet 6ad37afce1d6430ea1b03b4f5de92e1d (table test-table [id=aae9ae5023ab4ce68859158fe7148f36]) successfully deleted
I20260807 11:26:37.351215 529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:37.321496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52556 (local address 127.16.197.1:45945)
0807 11:26:37.322236 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:37.322245 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:37.322267 (+ 22us) server_negotiation.cc:407] Connection header received
0807 11:26:37.322336 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:37.322344 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:37.322452 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:37.322569 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:37.325165 (+ 2596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.326727 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.329329 (+ 2602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.330318 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.330493 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:37.337046 (+ 6553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:37.337075 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:37.337100 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:37.337167 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:37.342417 (+ 5250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:37.344125 (+ 1708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:37.344138 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:37.344145 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:37.344345 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:37.347532 (+ 3187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:37.347541 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:37.347547 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:37.348236 (+ 689us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:37.348636 (+ 400us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:37.350252 (+ 1616us) server_negotiation.cc:299] Negotiation successful
0807 11:26:37.350592 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575}
I20260807 11:26:37.352152 32697 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:26:37.358291 557 ts_tablet_manager.cc:1918] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:26:37.358664 557 log.cc:1196] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/wal/wals/6ad37afce1d6430ea1b03b4f5de92e1d
I20260807 11:26:37.359455 557 ts_tablet_manager.cc:1939] T 6ad37afce1d6430ea1b03b4f5de92e1d P 152888668c1a4d43bdc59dedb3facdd5: Deleting consensus metadata
I20260807 11:26:37.361660 32487 catalog_manager.cc:4841] TS 152888668c1a4d43bdc59dedb3facdd5 (127.16.197.3:45307): tablet 6ad37afce1d6430ea1b03b4f5de92e1d (table test-table [id=aae9ae5023ab4ce68859158fe7148f36]) successfully deleted
I20260807 11:26:37.472712 556 tablet_replica.cc:330] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee: stopping tablet replica
I20260807 11:26:37.473181 556 raft_consensus.cc:2229] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:26:37.473666 556 raft_consensus.cc:2258] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:26:37.474706 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 32544
I20260807 11:26:37.501267 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 32706
W20260807 11:26:37.501457 32489 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.197.1:45945 (error 108)
I20260807 11:26:37.504688 526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:37.504168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:45945 (local address 127.0.0.1:52572)
0807 11:26:37.504377 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:37.504506 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260807 11:26:37.505029 32489 catalog_manager.cc:4584] TS 0497e7bcd8254497bb2635405cde23ee (127.16.197.1:45945): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6ad37afce1d6430ea1b03b4f5de92e1d: Network error: Client connection negotiation failed: client connection to 127.16.197.1:45945: connect: Connection refused (error 111)
I20260807 11:26:37.521425 525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:37.502335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50796 (local address 127.16.197.2:35973)
0807 11:26:37.502525 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:37.502532 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:37.502556 (+ 24us) server_negotiation.cc:407] Connection header received
0807 11:26:37.502825 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:37.502831 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:37.502934 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:37.503080 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:37.504934 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.506090 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.507667 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.508236 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.508374 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:37.514263 (+ 5889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:37.514282 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:37.514302 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:37.514348 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:37.518403 (+ 4055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:37.519177 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:37.519187 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:37.519191 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:37.519353 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:37.519740 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:37.519746 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:37.519750 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:37.520264 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:37.520574 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:37.520797 (+ 223us) server_negotiation.cc:299] Negotiation successful
0807 11:26:37.521034 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 11:26:37.522198 369 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:26:37.558744 526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:37.558162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:45945 (local address 127.0.0.1:52574)
0807 11:26:37.558448 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:37.558559 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160}
I20260807 11:26:37.559947 526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:37.559550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:45945 (local address 127.0.0.1:52586)
0807 11:26:37.559713 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:37.559819 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 11:26:37.620811 526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:37.620237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:45945 (local address 127.0.0.1:52600)
0807 11:26:37.620509 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:37.620619 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260807 11:26:37.622108 526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:37.621659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:45945 (local address 127.0.0.1:52610)
0807 11:26:37.621844 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:37.621947 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 11:26:37.627318 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 32706
I20260807 11:26:37.653395 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 377
I20260807 11:26:37.668879 530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:37.654181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51968 (local address 127.16.197.3:45307)
0807 11:26:37.654384 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:37.654391 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:37.654472 (+ 81us) server_negotiation.cc:407] Connection header received
0807 11:26:37.654685 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:37.654691 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:37.654785 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:37.654922 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:37.656392 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.657219 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.658580 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:37.659171 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:37.659308 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:37.662136 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:37.662156 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:37.662172 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:37.662218 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:37.665703 (+ 3485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:37.666504 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:37.666519 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:37.666525 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:37.666700 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:37.667171 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:37.667176 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:37.667184 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:37.667552 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:37.667918 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:37.668226 (+ 308us) server_negotiation.cc:299] Negotiation successful
0807 11:26:37.668458 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 11:26:37.669540 510 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:26:37.703132 526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:37.702514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:45945 (local address 127.0.0.1:52624)
0807 11:26:37.702792 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:37.702890 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260807 11:26:37.704200 526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:37.703819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:45945 (local address 127.0.0.1:52626)
0807 11:26:37.704021 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:37.704097 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260807 11:26:37.796404 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 377
I20260807 11:26:37.819208 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 32470
I20260807 11:26:37.820441 32535 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:26:37.936805 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 32470
I20260807 11:26:37.936892 526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:37.936269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:45945 (local address 127.0.0.1:52636)
0807 11:26:37.936459 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:37.936573 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 11:26:37.961603 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:42873
--webserver_interface=127.16.197.62
--webserver_port=39769
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:36135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:42873
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:26:38.310425 563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:26:38.310750 563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:26:38.310885 563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:26:38.318512 563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:26:38.318637 563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:26:38.318686 563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:26:38.318787 563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:26:38.318866 563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:26:38.329362 563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:36135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:42873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:42873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=39769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:26:38.330844 563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:26:38.332249 563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:26:38.344209 572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:38.344154 570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:38.344235 569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:38.345132 563 server_base.cc:1034] running on GCE node
I20260807 11:26:38.345929 563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:26:38.347442 563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:26:38.348668 563 hybrid_clock.cc:648] HybridClock initialized: now 1786101998348608 us; error 53 us; skew 500 ppm
I20260807 11:26:38.349012 563 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:26:38 dist-test-slave-2w3w krb5kdc[31887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101998, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:26:38.352146 563 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:26:38.353708 563 webserver.cc:460] Webserver started at http://127.16.197.62:39769/ using document root <none> and password file <none>
I20260807 11:26:38.354386 563 fs_manager.cc:359] Metadata directory not provided
I20260807 11:26:38.354537 563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:26:38.360119 563 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260807 11:26:38.363023 580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:38.364316 563 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 11:26:38.364508 563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "ad8a83ed1c634f0eaf02452b9216f675"
format_stamp: "Formatted at 2026-08-07 11:26:32 on dist-test-slave-2w3w"
server_key: "b9b9d9f56e145c686e6c5d31a63d41ff"
server_key_iv: "31f960df17f3be57b24f0fc8cb28bdc1"
server_key_version: "encryptionkey@0"
I20260807 11:26:38.365502 563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:26:38.384914 563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:26:38.388832 563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:26:38.389127 563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:26:38.408654 563 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:42873
I20260807 11:26:38.408699 633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:42873 every 8 connection(s)
I20260807 11:26:38.410497 563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:26:38.417740 634 sys_catalog.cc:263] Verifying existing consensus state
I20260807 11:26:38.419188 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 563
I20260807 11:26:38.420293 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:45945
--local_ip_for_outbound_sockets=127.16.197.1
--tserver_master_addrs=127.16.197.62:42873
--webserver_port=35905
--webserver_interface=127.16.197.1
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:36135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 11:26:38.422726 634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675: Bootstrap starting.
I20260807 11:26:38.446429 634 log.cc:824] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675: Log is configured to *not* fsync() on all Append() calls
I20260807 11:26:38.463713 634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 11:26:38.464322 634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675: Bootstrap complete.
I20260807 11:26:38.474306 634 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8a83ed1c634f0eaf02452b9216f675" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42873 } }
I20260807 11:26:38.475378 634 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad8a83ed1c634f0eaf02452b9216f675, State: Initialized, Role: FOLLOWER
I20260807 11:26:38.476069 634 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8a83ed1c634f0eaf02452b9216f675" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42873 } }
I20260807 11:26:38.476320 634 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:26:38.476447 634 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:26:38.476646 634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [term 2 FOLLOWER]: Advancing to term 3
I20260807 11:26:38.480916 634 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8a83ed1c634f0eaf02452b9216f675" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42873 } }
I20260807 11:26:38.481489 634 leader_election.cc:302] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad8a83ed1c634f0eaf02452b9216f675; no voters:
I20260807 11:26:38.482218 634 leader_election.cc:288] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 11:26:38.482503 638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 11:26:38.483615 638 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [term 3 LEADER]: Becoming Leader. State: Replica: ad8a83ed1c634f0eaf02452b9216f675, State: Running, Role: LEADER
I20260807 11:26:38.484234 638 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8a83ed1c634f0eaf02452b9216f675" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42873 } }
I20260807 11:26:38.484683 634 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:26:38.487238 639 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ad8a83ed1c634f0eaf02452b9216f675" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8a83ed1c634f0eaf02452b9216f675" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42873 } } }
I20260807 11:26:38.487149 640 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad8a83ed1c634f0eaf02452b9216f675. Latest consensus state: current_term: 3 leader_uuid: "ad8a83ed1c634f0eaf02452b9216f675" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8a83ed1c634f0eaf02452b9216f675" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 42873 } } }
I20260807 11:26:38.487541 640 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [sys.catalog]: This master's current role is: LEADER
I20260807 11:26:38.487540 639 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad8a83ed1c634f0eaf02452b9216f675 [sys.catalog]: This master's current role is: LEADER
I20260807 11:26:38.497270 644 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:26:38.506466 644 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:26:38.508546 644 catalog_manager.cc:1259] Loaded cluster ID: 3d5eb72df9f2420db50028e09f26da9c
I20260807 11:26:38.508701 644 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 11:26:38.511608 644 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc4968829ad google::LogMessage::Fail() at ??:0
@ 0x7fc49688588a google::LogMessage::SendToLog() at ??:0
@ 0x7fc4968824de google::LogMessage::Flush() at ??:0
@ 0x7fc496883a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc4ac2b5512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fc4ac2b2976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc4ac33a610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fc4ac33a439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc4994ce345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc4994d191c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fc4994d1799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc4994aceaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc49c7866db start_thread at ??:0
@ 0x7fc49571c61f clone at ??:0
W20260807 11:26:38.811750 636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:26:38.812054 636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:26:38.812155 636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:26:38.820075 636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:26:38.820220 636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:26:38.820353 636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:26:38.830488 636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:36135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:45945
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=35905
--tserver_master_addrs=127.16.197.62:42873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:26:38.831946 636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:26:38.833324 636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:26:38.847405 678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:38.847316 681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:38.847340 679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:38.848374 636 server_base.cc:1034] running on GCE node
I20260807 11:26:38.849017 636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:26:38.849941 636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:26:38.851141 636 hybrid_clock.cc:648] HybridClock initialized: now 1786101998851091 us; error 36 us; skew 500 ppm
I20260807 11:26:38.851475 636 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:26:38 dist-test-slave-2w3w krb5kdc[31887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101998, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:26:38.854807 636 init.cc:377] Logged in from keytab as kudu/127.16.197.1@KRBTEST.COM (short username kudu)
I20260807 11:26:38.856398 636 webserver.cc:460] Webserver started at http://127.16.197.1:35905/ using document root <none> and password file <none>
I20260807 11:26:38.857137 636 fs_manager.cc:359] Metadata directory not provided
I20260807 11:26:38.857287 636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:26:38.863145 636 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260807 11:26:38.866109 689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:38.867599 636 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:26:38.867797 636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "0497e7bcd8254497bb2635405cde23ee"
format_stamp: "Formatted at 2026-08-07 11:26:32 on dist-test-slave-2w3w"
server_key: "21a65431e8ba46ba810568ddc6342b46"
server_key_iv: "f5a48fc07a961cc0679ce8cb02d33032"
server_key_version: "encryptionkey@0"
I20260807 11:26:38.868866 636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:26:38.889403 636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:26:38.894153 636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:26:38.894485 636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:26:38.895814 636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:26:38.897710 696 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260807 11:26:38.903844 636 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 11:26:38.904035 636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260807 11:26:38.904191 636 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 11:26:38.907800 636 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 11:26:38.907923 636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.003s sys 0.000s
I20260807 11:26:38.908344 696 tablet_bootstrap.cc:492] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee: Bootstrap starting.
I20260807 11:26:38.918728 696 log.cc:824] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee: Log is configured to *not* fsync() on all Append() calls
I20260807 11:26:38.936688 696 tablet_bootstrap.cc:492] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 11:26:38.937412 696 tablet_bootstrap.cc:492] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee: Bootstrap complete.
I20260807 11:26:38.937953 696 ts_tablet_manager.cc:1397] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee: Time spent bootstrapping tablet: real 0.030s user 0.023s sys 0.004s
I20260807 11:26:38.949240 696 raft_consensus.cc:348] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6174176c30e34fd1afb4529ac87df331" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35973 } } peers { permanent_uuid: "0497e7bcd8254497bb2635405cde23ee" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45945 } } peers { permanent_uuid: "152888668c1a4d43bdc59dedb3facdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45307 } }
I20260807 11:26:38.951074 696 raft_consensus.cc:729] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0497e7bcd8254497bb2635405cde23ee, State: Initialized, Role: FOLLOWER
I20260807 11:26:38.951810 696 consensus_queue.cc:260] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6174176c30e34fd1afb4529ac87df331" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35973 } } peers { permanent_uuid: "0497e7bcd8254497bb2635405cde23ee" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45945 } } peers { permanent_uuid: "152888668c1a4d43bdc59dedb3facdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45307 } }
I20260807 11:26:38.953429 636 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:45945
I20260807 11:26:38.953528 812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:45945 every 8 connection(s)
I20260807 11:26:38.954855 636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:26:38.956733 696 ts_tablet_manager.cc:1428] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee: Time spent starting tablet: real 0.019s user 0.012s sys 0.008s
I20260807 11:26:38.960211 815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:38.957994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.1:33885)
0807 11:26:38.958826 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:38.959164 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":365,"threads_started":1}
W20260807 11:26:38.960947 813 heartbeater.cc:644] Failed to heartbeat to 127.16.197.62:42873 (0 consecutive failures): Network error: Failed to ping master at 127.16.197.62:42873: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
I20260807 11:26:38.961902 815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:38.961504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.1:39935)
0807 11:26:38.961669 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:38.961762 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 11:26:38.962810 815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:38.962444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.1:43009)
0807 11:26:38.962607 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:38.962682 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 11:26:38.963164 813 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 11:26:38.963406 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 636
I20260807 11:26:38.964082 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:35973
--local_ip_for_outbound_sockets=127.16.197.2
--tserver_master_addrs=127.16.197.62:42873
--webserver_port=38967
--webserver_interface=127.16.197.2
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:36135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:26:39.315073 817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:26:39.315382 817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:26:39.315487 817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:26:39.322881 817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:26:39.323026 817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:26:39.323148 817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:26:39.333149 817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:36135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:35973
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=38967
--tserver_master_addrs=127.16.197.62:42873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:26:39.334460 817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:26:39.335808 817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:26:39.349627 822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:39.349841 825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:39.349659 823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:39.350728 817 server_base.cc:1034] running on GCE node
I20260807 11:26:39.351466 817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:26:39.352500 817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:26:39.353781 817 hybrid_clock.cc:648] HybridClock initialized: now 1786101999353717 us; error 55 us; skew 500 ppm
I20260807 11:26:39.354137 817 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:26:39 dist-test-slave-2w3w krb5kdc[31887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101999, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:26:39.357731 817 init.cc:377] Logged in from keytab as kudu/127.16.197.2@KRBTEST.COM (short username kudu)
I20260807 11:26:39.359570 817 webserver.cc:460] Webserver started at http://127.16.197.2:38967/ using document root <none> and password file <none>
I20260807 11:26:39.360402 817 fs_manager.cc:359] Metadata directory not provided
I20260807 11:26:39.360559 817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:26:39.366632 817 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 11:26:39.369534 832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:39.371301 817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 11:26:39.371502 817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "6174176c30e34fd1afb4529ac87df331"
format_stamp: "Formatted at 2026-08-07 11:26:33 on dist-test-slave-2w3w"
server_key: "29c20b461afe09c2275938a988d709c9"
server_key_iv: "b9a21728bf26fbf0bfa0d98598a5bc97"
server_key_version: "encryptionkey@0"
I20260807 11:26:39.372593 817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:26:39.389267 817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:26:39.393819 817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:26:39.394074 817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:26:39.395186 817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:26:39.396745 817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:26:39.396848 817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:39.396929 817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:26:39.396983 817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:39.433846 817 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:35973
I20260807 11:26:39.433938 948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:35973 every 8 connection(s)
I20260807 11:26:39.435459 817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:26:39.440408 951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:39.438453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.2:33759)
0807 11:26:39.439280 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:39.439498 (+ 218us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":310,"threads_started":1}
W20260807 11:26:39.441186 949 heartbeater.cc:644] Failed to heartbeat to 127.16.197.62:42873 (0 consecutive failures): Network error: Failed to ping master at 127.16.197.62:42873: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
I20260807 11:26:39.442093 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 817
I20260807 11:26:39.442162 951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:39.441694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.2:40599)
0807 11:26:39.441913 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:39.442015 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 11:26:39.442962 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:45307
--local_ip_for_outbound_sockets=127.16.197.3
--tserver_master_addrs=127.16.197.62:42873
--webserver_port=36869
--webserver_interface=127.16.197.3
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:36135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 11:26:39.443285 951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:39.442764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.2:44985)
0807 11:26:39.442938 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:39.443114 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260807 11:26:39.443593 949 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 11:26:39.798727 953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:26:39.799122 953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:26:39.799312 953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:26:39.806897 953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:26:39.807144 953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:26:39.807336 953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:26:39.817672 953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:36135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:45307
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=36869
--tserver_master_addrs=127.16.197.62:42873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:26:39.819132 953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:26:39.820504 953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:26:39.834470 958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:39.834417 961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:39.834443 959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:39.835238 953 server_base.cc:1034] running on GCE node
I20260807 11:26:39.835932 953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:26:39.836964 953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:26:39.838191 953 hybrid_clock.cc:648] HybridClock initialized: now 1786101999838119 us; error 56 us; skew 500 ppm
I20260807 11:26:39.838557 953 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:26:39 dist-test-slave-2w3w krb5kdc[31887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101999, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:26:39.842103 953 init.cc:377] Logged in from keytab as kudu/127.16.197.3@KRBTEST.COM (short username kudu)
I20260807 11:26:39.843791 953 webserver.cc:460] Webserver started at http://127.16.197.3:36869/ using document root <none> and password file <none>
I20260807 11:26:39.844506 953 fs_manager.cc:359] Metadata directory not provided
I20260807 11:26:39.844619 953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:26:39.850507 953 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260807 11:26:39.853639 969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:39.855283 953 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 11:26:39.855476 953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "152888668c1a4d43bdc59dedb3facdd5"
format_stamp: "Formatted at 2026-08-07 11:26:34 on dist-test-slave-2w3w"
server_key: "b9c421176e38932e8ddd7827d62b9be9"
server_key_iv: "712fb01d6d4a437b94f68ce11ee62167"
server_key_version: "encryptionkey@0"
I20260807 11:26:39.856561 953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:26:39.878515 953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:26:39.882776 953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:26:39.883143 953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:26:39.884228 953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:26:39.885708 953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:26:39.885856 953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:39.885993 953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:26:39.886081 953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:39.921082 953 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:45307
I20260807 11:26:39.921172 1085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:45307 every 8 connection(s)
I20260807 11:26:39.922659 953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:26:39.928090 1088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:39.926050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.3:38143)
0807 11:26:39.926865 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:39.927162 (+ 297us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":308,"threads_started":1}
W20260807 11:26:39.928977 1086 heartbeater.cc:644] Failed to heartbeat to 127.16.197.62:42873 (0 consecutive failures): Network error: Failed to ping master at 127.16.197.62:42873: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
I20260807 11:26:39.929929 1088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:39.929563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.3:46801)
0807 11:26:39.929724 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:39.929803 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 11:26:39.930877 1088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:39.930514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.3:52729)
0807 11:26:39.930672 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:39.930759 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 11:26:39.931207 1086 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 11:26:39.931905 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 953
I20260807 11:26:39.964767 1091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:39.963782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.1:43683)
0807 11:26:39.964416 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:39.964561 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":361,"threads_started":1}
I20260807 11:26:40.197495 1092 raft_consensus.cc:482] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:26:40.197795 1092 raft_consensus.cc:504] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6174176c30e34fd1afb4529ac87df331" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35973 } } peers { permanent_uuid: "0497e7bcd8254497bb2635405cde23ee" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45945 } } peers { permanent_uuid: "152888668c1a4d43bdc59dedb3facdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45307 } }
I20260807 11:26:40.199630 1092 leader_election.cc:288] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6174176c30e34fd1afb4529ac87df331 (127.16.197.2:35973), 152888668c1a4d43bdc59dedb3facdd5 (127.16.197.3:45307)
Aug 07 11:26:40 dist-test-slave-2w3w krb5kdc[31887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101998, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for kudu/127.16.197.2@KRBTEST.COM
Aug 07 11:26:40 dist-test-slave-2w3w krb5kdc[31887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786101998, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for kudu/127.16.197.3@KRBTEST.COM
I20260807 11:26:40.222322 1093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:40.200023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:35973 (local address 127.16.197.1:37807)
0807 11:26:40.202746 (+ 2723us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:40.202762 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:40.202928 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:40.205636 (+ 2708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:40.205640 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:40.206185 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:26:40.206762 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:40.206781 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:40.208380 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:40.208384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:40.209909 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:40.209921 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:40.210915 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:40.210921 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:40.211263 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:40.212120 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:26:40.212137 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:26:40.214786 (+ 2649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:26:40.218650 (+ 3864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:40.218660 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:40.218677 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:40.219213 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:40.219656 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:40.219662 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:40.219666 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:40.219874 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:40.220669 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:26:40.220677 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:26:40.220990 (+ 313us) client_negotiation.cc:769] Sending connection context
0807 11:26:40.221267 (+ 277us) client_negotiation.cc:240] Negotiation successful
0807 11:26:40.221834 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2588,"mutex_wait_us":254,"spinlock_wait_cycles":9472,"thread_start_us":2411,"threads_started":1}
I20260807 11:26:40.222314 1094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:40.200289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:37807 (local address 127.16.197.2:35973)
0807 11:26:40.202385 (+ 2096us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:40.202402 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:40.202895 (+ 493us) server_negotiation.cc:407] Connection header received
0807 11:26:40.204884 (+ 1989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:40.204897 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:40.205322 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:40.205687 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:40.206985 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:40.208180 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:40.210065 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:40.210685 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:40.210900 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:40.215230 (+ 4330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:40.215245 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:40.215265 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:40.215336 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:40.218498 (+ 3162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:40.219358 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:40.219372 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:40.219377 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:40.219540 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:40.219998 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:40.220002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:40.220004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:40.220517 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:40.220718 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:40.221515 (+ 797us) server_negotiation.cc:299] Negotiation successful
0807 11:26:40.221898 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1900,"thread_start_us":1762,"threads_started":1}
I20260807 11:26:40.222322 1095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:40.200396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:57373 (local address 127.16.197.3:45307)
0807 11:26:40.202195 (+ 1799us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:40.202216 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:40.202274 (+ 58us) server_negotiation.cc:407] Connection header received
0807 11:26:40.204044 (+ 1770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:40.204058 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:40.204548 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:40.204901 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:40.207035 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:40.208185 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:40.210040 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:40.210684 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:40.210893 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:40.215313 (+ 4420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:40.215320 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:40.215333 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:40.215375 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:40.218634 (+ 3259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:40.219429 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:40.219435 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:40.219440 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:40.219597 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:40.219988 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:40.219993 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:40.219996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:40.220527 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:40.220732 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:40.221523 (+ 791us) server_negotiation.cc:299] Negotiation successful
0807 11:26:40.221898 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1533,"thread_start_us":1276,"threads_started":1}
I20260807 11:26:40.222402 1091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:40.199975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:45307 (local address 127.16.197.1:57373)
0807 11:26:40.200333 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:40.200395 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:40.201098 (+ 703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:40.205089 (+ 3991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:40.205101 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:40.205994 (+ 893us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:26:40.206759 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:40.206782 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:40.208389 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:40.208393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:40.209888 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:40.209908 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:40.210942 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:40.210949 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:40.211261 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:40.212068 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:26:40.212091 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:26:40.215030 (+ 2939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:26:40.218802 (+ 3772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:40.218807 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:40.218811 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:40.219288 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:40.219720 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:40.219727 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:40.219731 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:40.219878 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:40.220669 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:26:40.220677 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:26:40.220996 (+ 319us) client_negotiation.cc:769] Sending connection context
0807 11:26:40.221272 (+ 276us) client_negotiation.cc:240] Negotiation successful
0807 11:26:40.221986 (+ 714us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":361,"spinlock_wait_cycles":30592}
W20260807 11:26:40.225739 690 leader_election.cc:341] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 152888668c1a4d43bdc59dedb3facdd5 (127.16.197.3:45307): Not found: Tablet not found: 6ad37afce1d6430ea1b03b4f5de92e1d
W20260807 11:26:40.226727 691 leader_election.cc:341] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6174176c30e34fd1afb4529ac87df331 (127.16.197.2:35973): Not found: Tablet not found: 6ad37afce1d6430ea1b03b4f5de92e1d
I20260807 11:26:40.226891 691 leader_election.cc:302] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0497e7bcd8254497bb2635405cde23ee; no voters: 152888668c1a4d43bdc59dedb3facdd5, 6174176c30e34fd1afb4529ac87df331
I20260807 11:26:40.227402 1092 raft_consensus.cc:2726] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 11:26:40.445103 1096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:40.444078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.2:38337)
0807 11:26:40.444671 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:40.444857 (+ 186us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":297,"threads_started":1}
I20260807 11:26:40.932596 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:40.931802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.3:44563)
0807 11:26:40.932262 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:40.932398 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":219,"threads_started":1}
I20260807 11:26:40.966701 1098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:40.965912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.1:43929)
0807 11:26:40.966410 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:40.966503 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":249,"threads_started":1}
I20260807 11:26:41.446861 1099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:41.446029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.2:42001)
0807 11:26:41.446579 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:41.446688 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":305,"threads_started":1}
I20260807 11:26:41.934455 1100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:41.933688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.3:52229)
0807 11:26:41.934167 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:41.934255 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":250,"threads_started":1}
I20260807 11:26:41.968547 1101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:41.967705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.1:52239)
0807 11:26:41.968224 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:41.968333 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":255,"threads_started":1}
I20260807 11:26:42.340862 1102 raft_consensus.cc:482] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:26:42.341068 1102 raft_consensus.cc:504] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6174176c30e34fd1afb4529ac87df331" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35973 } } peers { permanent_uuid: "0497e7bcd8254497bb2635405cde23ee" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45945 } } peers { permanent_uuid: "152888668c1a4d43bdc59dedb3facdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45307 } }
I20260807 11:26:42.341743 1102 leader_election.cc:288] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6174176c30e34fd1afb4529ac87df331 (127.16.197.2:35973), 152888668c1a4d43bdc59dedb3facdd5 (127.16.197.3:45307)
W20260807 11:26:42.342737 691 leader_election.cc:341] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6174176c30e34fd1afb4529ac87df331 (127.16.197.2:35973): Not found: Tablet not found: 6ad37afce1d6430ea1b03b4f5de92e1d
W20260807 11:26:42.342902 690 leader_election.cc:341] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 152888668c1a4d43bdc59dedb3facdd5 (127.16.197.3:45307): Not found: Tablet not found: 6ad37afce1d6430ea1b03b4f5de92e1d
I20260807 11:26:42.343191 690 leader_election.cc:302] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0497e7bcd8254497bb2635405cde23ee; no voters: 152888668c1a4d43bdc59dedb3facdd5, 6174176c30e34fd1afb4529ac87df331
I20260807 11:26:42.343461 1102 raft_consensus.cc:2726] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 11:26:42.448465 1103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:42.447744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.2:39645)
0807 11:26:42.448194 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:42.448294 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":235,"threads_started":1}
I20260807 11:26:42.936048 1104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:42.935354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.3:53193)
0807 11:26:42.935796 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:42.935887 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":222,"threads_started":1}
I20260807 11:26:42.970310 1105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:42.969569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.1:49601)
0807 11:26:42.970060 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:42.970145 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":266,"threads_started":1}
I20260807 11:26:43.450119 1106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:43.449348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.2:56289)
0807 11:26:43.449833 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:43.449928 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":226,"threads_started":1}
I20260807 11:26:43.937750 1107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:43.936988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.3:35373)
0807 11:26:43.937477 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:43.937571 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":239,"threads_started":1}
I20260807 11:26:43.972478 1109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:43.971548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.1:32875)
0807 11:26:43.972127 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:43.972274 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":264,"threads_started":1}
I20260807 11:26:44.152045 1110 raft_consensus.cc:482] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:26:44.152280 1110 raft_consensus.cc:504] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6174176c30e34fd1afb4529ac87df331" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35973 } } peers { permanent_uuid: "0497e7bcd8254497bb2635405cde23ee" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45945 } } peers { permanent_uuid: "152888668c1a4d43bdc59dedb3facdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45307 } }
I20260807 11:26:44.153090 1110 leader_election.cc:288] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6174176c30e34fd1afb4529ac87df331 (127.16.197.2:35973), 152888668c1a4d43bdc59dedb3facdd5 (127.16.197.3:45307)
W20260807 11:26:44.154142 690 leader_election.cc:341] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 152888668c1a4d43bdc59dedb3facdd5 (127.16.197.3:45307): Not found: Tablet not found: 6ad37afce1d6430ea1b03b4f5de92e1d
W20260807 11:26:44.154347 691 leader_election.cc:341] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6174176c30e34fd1afb4529ac87df331 (127.16.197.2:35973): Not found: Tablet not found: 6ad37afce1d6430ea1b03b4f5de92e1d
I20260807 11:26:44.154495 691 leader_election.cc:302] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0497e7bcd8254497bb2635405cde23ee; no voters: 152888668c1a4d43bdc59dedb3facdd5, 6174176c30e34fd1afb4529ac87df331
I20260807 11:26:44.154721 1110 raft_consensus.cc:2726] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 11:26:44.451879 1111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:44.451126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.2:50081)
0807 11:26:44.451615 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:44.451698 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":270,"threads_started":1}
I20260807 11:26:44.939567 1112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:44.938804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.3:49243)
0807 11:26:44.939311 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:44.939406 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":289,"threads_started":1}
I20260807 11:26:44.974125 1113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:44.973326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.1:41859)
0807 11:26:44.973798 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:44.973891 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":257,"threads_started":1}
I20260807 11:26:45.453557 1114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:45.452781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.2:53877)
0807 11:26:45.453281 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:45.453373 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":257,"threads_started":1}
I20260807 11:26:45.941452 1115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:45.940584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.3:52353)
0807 11:26:45.941154 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:45.941249 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":323,"threads_started":1}
I20260807 11:26:45.976063 1117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:45.975197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.1:42249)
0807 11:26:45.975697 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:45.975806 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":247,"threads_started":1}
I20260807 11:26:46.455732 1118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:46.454735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.2:52047)
0807 11:26:46.455458 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:46.455541 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":339,"threads_started":1}
I20260807 11:26:46.943384 1119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:46.942390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.3:46635)
0807 11:26:46.943072 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:46.943162 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":291,"threads_started":1}
I20260807 11:26:46.977844 1120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:46.977036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.1:37627)
0807 11:26:46.977564 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:46.977658 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":234,"threads_started":1}
I20260807 11:26:47.457700 1121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:47.456932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.2:38379)
0807 11:26:47.457412 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:47.457503 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":251,"threads_started":1}
I20260807 11:26:47.945358 1122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:47.944512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.3:51481)
0807 11:26:47.945056 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:47.945154 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":277,"threads_started":1}
I20260807 11:26:47.979625 1123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:47.978800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.1:38539)
0807 11:26:47.979344 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:47.979440 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":290,"threads_started":1}
I20260807 11:26:48.048255 1124 raft_consensus.cc:482] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:26:48.048525 1124 raft_consensus.cc:504] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6174176c30e34fd1afb4529ac87df331" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35973 } } peers { permanent_uuid: "0497e7bcd8254497bb2635405cde23ee" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45945 } } peers { permanent_uuid: "152888668c1a4d43bdc59dedb3facdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45307 } }
I20260807 11:26:48.049358 1124 leader_election.cc:288] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6174176c30e34fd1afb4529ac87df331 (127.16.197.2:35973), 152888668c1a4d43bdc59dedb3facdd5 (127.16.197.3:45307)
W20260807 11:26:48.050302 691 leader_election.cc:341] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6174176c30e34fd1afb4529ac87df331 (127.16.197.2:35973): Not found: Tablet not found: 6ad37afce1d6430ea1b03b4f5de92e1d
W20260807 11:26:48.050509 690 leader_election.cc:341] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 152888668c1a4d43bdc59dedb3facdd5 (127.16.197.3:45307): Not found: Tablet not found: 6ad37afce1d6430ea1b03b4f5de92e1d
I20260807 11:26:48.050643 690 leader_election.cc:302] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0497e7bcd8254497bb2635405cde23ee; no voters: 152888668c1a4d43bdc59dedb3facdd5, 6174176c30e34fd1afb4529ac87df331
I20260807 11:26:48.050941 1124 raft_consensus.cc:2726] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 11:26:48.459585 1125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:48.458728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.2:58243)
0807 11:26:48.459323 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:48.459413 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":329,"threads_started":1}
I20260807 11:26:48.947177 1126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:48.946352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.3:43115)
0807 11:26:48.946874 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:48.947005 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":243,"threads_started":1}
I20260807 11:26:48.981485 1127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:48.980639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.1:60579)
0807 11:26:48.981168 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:48.981281 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":283,"threads_started":1}
I20260807 11:26:49.461625 1128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:49.460639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.2:48927)
0807 11:26:49.461211 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:49.461382 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":258,"threads_started":1}
I20260807 11:26:49.949101 1129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:49.948287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.3:57125)
0807 11:26:49.948814 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:49.948911 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":277,"threads_started":1}
I20260807 11:26:49.983350 1130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:49.982514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.1:59029)
0807 11:26:49.983068 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:49.983167 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":254,"threads_started":1}
I20260807 11:26:50.463495 1131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:50.462655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.2:57295)
0807 11:26:50.463234 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:50.463329 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":250,"threads_started":1}
I20260807 11:26:50.951069 1132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:50.950137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.3:50677)
0807 11:26:50.950730 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:50.950838 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":286,"threads_started":1}
I20260807 11:26:50.985188 1134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:50.984328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.1:59971)
0807 11:26:50.984925 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:50.985011 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":328,"threads_started":1}
I20260807 11:26:51.465293 1136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:51.464511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.2:33035)
0807 11:26:51.465028 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:51.465120 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":237,"threads_started":1}
I20260807 11:26:51.764104 1137 raft_consensus.cc:482] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:26:51.764287 1137 raft_consensus.cc:504] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6174176c30e34fd1afb4529ac87df331" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35973 } } peers { permanent_uuid: "0497e7bcd8254497bb2635405cde23ee" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 45945 } } peers { permanent_uuid: "152888668c1a4d43bdc59dedb3facdd5" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45307 } }
I20260807 11:26:51.764883 1137 leader_election.cc:288] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6174176c30e34fd1afb4529ac87df331 (127.16.197.2:35973), 152888668c1a4d43bdc59dedb3facdd5 (127.16.197.3:45307)
W20260807 11:26:51.765966 691 leader_election.cc:341] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6174176c30e34fd1afb4529ac87df331 (127.16.197.2:35973): Not found: Tablet not found: 6ad37afce1d6430ea1b03b4f5de92e1d
W20260807 11:26:51.766230 690 leader_election.cc:341] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 152888668c1a4d43bdc59dedb3facdd5 (127.16.197.3:45307): Not found: Tablet not found: 6ad37afce1d6430ea1b03b4f5de92e1d
I20260807 11:26:51.766384 690 leader_election.cc:302] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0497e7bcd8254497bb2635405cde23ee; no voters: 152888668c1a4d43bdc59dedb3facdd5, 6174176c30e34fd1afb4529ac87df331
I20260807 11:26:51.766687 1137 raft_consensus.cc:2726] T 6ad37afce1d6430ea1b03b4f5de92e1d P 0497e7bcd8254497bb2635405cde23ee [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 11:26:51.952843 1139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:51.952068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.3:56937)
0807 11:26:51.952582 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:51.952666 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":272,"threads_started":1}
I20260807 11:26:51.986908 1140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:51.986103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.1:35085)
0807 11:26:51.986589 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:51.986687 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":264,"threads_started":1}
I20260807 11:26:52.467319 1141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:52.466384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.2:33177)
0807 11:26:52.466897 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:52.467080 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":249,"threads_started":1}
I20260807 11:26:52.954874 1142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:52.953897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.3:52397)
0807 11:26:52.954562 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:52.954670 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":364,"threads_started":1}
I20260807 11:26:52.988889 1143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:52.988070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.1:37399)
0807 11:26:52.988580 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:52.988690 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":277,"threads_started":1}
I20260807 11:26:53.469086 1144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:53.468306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.2:49407)
0807 11:26:53.468799 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:53.468883 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":242,"threads_started":1}
I20260807 11:26:53.956782 1145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:53.955885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.3:48253)
0807 11:26:53.956411 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:53.956522 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":251,"threads_started":1}
I20260807 11:26:53.991084 1146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:53.990136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.1:33563)
0807 11:26:53.990721 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:53.990838 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":320,"threads_started":1}
I20260807 11:26:54.471025 1147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:54.470194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.2:53451)
0807 11:26:54.470669 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:54.470829 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":236,"threads_started":1}
I20260807 11:26:54.940120 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 636
I20260807 11:26:54.956696 1148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:54.941452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58026 (local address 127.16.197.1:45945)
0807 11:26:54.942028 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:54.942036 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:54.942062 (+ 26us) server_negotiation.cc:407] Connection header received
0807 11:26:54.942125 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:54.942132 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:54.942373 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:54.942553 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:54.944231 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:54.945238 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:54.946574 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:54.947226 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:54.947385 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:54.950340 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:54.950353 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:54.950367 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:54.950401 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:54.953315 (+ 2914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:54.954107 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:54.954115 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:54.954117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:54.954267 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:54.954800 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:54.954807 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:54.954814 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:54.955277 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:54.955618 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:54.956028 (+ 410us) server_negotiation.cc:299] Negotiation successful
0807 11:26:54.956362 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":261,"threads_started":1}
I20260807 11:26:54.957990 807 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:26:54.959529 1149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:54.957997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:42873 (local address 127.16.197.3:52353)
0807 11:26:54.959182 (+ 1185us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:54.959337 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:42873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1057,"thread_start_us":968,"threads_started":1}
I20260807 11:26:55.051155 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 636
I20260807 11:26:55.071650 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 817
I20260807 11:26:55.086239 1152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:55.072497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45608 (local address 127.16.197.2:35973)
0807 11:26:55.073019 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:55.073023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:55.073039 (+ 16us) server_negotiation.cc:407] Connection header received
0807 11:26:55.073093 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:55.073097 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:55.073155 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:55.073278 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:55.074751 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:55.075643 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:55.077060 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:55.077479 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:55.077638 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:55.080346 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:55.080352 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:55.080355 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:55.080388 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:55.082926 (+ 2538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:55.083905 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:55.083913 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:55.083918 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:55.084062 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:55.084577 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:55.084583 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:55.084586 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:55.084964 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:55.085274 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:55.085592 (+ 318us) server_negotiation.cc:299] Negotiation successful
0807 11:26:55.085842 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":273,"threads_started":1}
I20260807 11:26:55.087069 944 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:26:55.172943 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 817
I20260807 11:26:55.194336 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 953
I20260807 11:26:55.209543 1154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:55.195382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36458 (local address 127.16.197.3:45307)
0807 11:26:55.196052 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:55.196057 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:55.196076 (+ 19us) server_negotiation.cc:407] Connection header received
0807 11:26:55.196131 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:55.196134 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:55.196203 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:55.196299 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:55.197754 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:55.198589 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:55.200149 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:55.200593 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:55.200693 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:55.203839 (+ 3146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:55.203845 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:55.203848 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:55.203882 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:55.206635 (+ 2753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:55.207521 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:55.207526 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:55.207528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:55.207688 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:55.208097 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:55.208101 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:55.208103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:55.208398 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:55.208629 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:55.208964 (+ 335us) server_negotiation.cc:299] Negotiation successful
0807 11:26:55.209140 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":333,"threads_started":1}
I20260807 11:26:55.210386 1081 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:26:55.296039 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 953
2026-08-07T11:26:55Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24749 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:26:55 dist-test-slave-2w3w krb5kdc[1158](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:26:55 dist-test-slave-2w3w krb5kdc[1158](info): set up 2 sockets
Aug 07 11:26:55 dist-test-slave-2w3w krb5kdc[1158](info): commencing operation
krb5kdc: starting...
W20260807 11:26:56.441908 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.004s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:26:56 dist-test-slave-2w3w krb5kdc[1158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:26:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:26:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:26:56.722532 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:46493
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:45231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:46493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:26:57.066152 1171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:26:57.066495 1171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:26:57.066644 1171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:26:57.073982 1171 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 11:26:57.074112 1171 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 11:26:57.074208 1171 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 11:26:57.074302 1171 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:26:57.074394 1171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:26:57.074508 1171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:26:57.074590 1171 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:26:57.074669 1171 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:26:57.084672 1171 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:45231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:46493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:46493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:26:57.086067 1171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:26:57.087538 1171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:26:57.099921 1177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:57.099939 1179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:57.100021 1176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:57.100411 1171 server_base.cc:1034] running on GCE node
I20260807 11:26:57.101415 1171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:26:57.102843 1171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:26:57.104111 1171 hybrid_clock.cc:648] HybridClock initialized: now 1786102017104032 us; error 56 us; skew 500 ppm
I20260807 11:26:57.104470 1171 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:26:57 dist-test-slave-2w3w krb5kdc[1158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102017, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:26:57.112480 1171 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:26:57.114051 1171 webserver.cc:460] Webserver started at http://127.16.197.62:43227/ using document root <none> and password file <none>
I20260807 11:26:57.114810 1171 fs_manager.cc:359] Metadata directory not provided
I20260807 11:26:57.114996 1171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:26:57.115345 1171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:26:57.118580 1171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "818d4fd98b34472384e0e9d8bdf31272"
format_stamp: "Formatted at 2026-08-07 11:26:57 on dist-test-slave-2w3w"
server_key: "018456d574668f87b119ebe7c05d051e"
server_key_iv: "a27613eb1c58dac2ebe84e6d2e3ae5e9"
server_key_version: "encryptionkey@0"
I20260807 11:26:57.119415 1171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "818d4fd98b34472384e0e9d8bdf31272"
format_stamp: "Formatted at 2026-08-07 11:26:57 on dist-test-slave-2w3w"
server_key: "018456d574668f87b119ebe7c05d051e"
server_key_iv: "a27613eb1c58dac2ebe84e6d2e3ae5e9"
server_key_version: "encryptionkey@0"
I20260807 11:26:57.124799 1171 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260807 11:26:57.128670 1186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:57.130131 1171 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 11:26:57.130318 1171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "818d4fd98b34472384e0e9d8bdf31272"
format_stamp: "Formatted at 2026-08-07 11:26:57 on dist-test-slave-2w3w"
server_key: "018456d574668f87b119ebe7c05d051e"
server_key_iv: "a27613eb1c58dac2ebe84e6d2e3ae5e9"
server_key_version: "encryptionkey@0"
I20260807 11:26:57.130553 1171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:26:57.155288 1171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:26:57.159729 1171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:26:57.160036 1171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:26:57.179347 1171 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:46493
I20260807 11:26:57.179443 1241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:46493 every 8 connection(s)
I20260807 11:26:57.180773 1171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:26:57.184544 1242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:26:57.189980 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 1171
I20260807 11:26:57.190208 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:26:57.191066 17172 external_mini_cluster.cc:1428] Setting key 2bae7cff5e4ca5ad9b33c1cdea772f34
I20260807 11:26:57.196722 1242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272: Bootstrap starting.
I20260807 11:26:57.199936 1242 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272: Neither blocks nor log segments found. Creating new log.
I20260807 11:26:57.200989 1242 log.cc:824] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272: Log is configured to *not* fsync() on all Append() calls
I20260807 11:26:57.204088 1242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272: No bootstrap required, opened a new log
Aug 07 11:26:57 dist-test-slave-2w3w krb5kdc[1158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:26:57.213398 1242 raft_consensus.cc:348] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818d4fd98b34472384e0e9d8bdf31272" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46493 } }
I20260807 11:26:57.213923 1242 raft_consensus.cc:374] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:26:57.214077 1242 raft_consensus.cc:729] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 818d4fd98b34472384e0e9d8bdf31272, State: Initialized, Role: FOLLOWER
I20260807 11:26:57.214763 1242 consensus_queue.cc:260] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818d4fd98b34472384e0e9d8bdf31272" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46493 } }
I20260807 11:26:57.215044 1242 raft_consensus.cc:388] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:26:57.215201 1242 raft_consensus.cc:482] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:26:57.215410 1242 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:26:57.215456 1245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:57.193414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55306 (local address 127.16.197.62:46493)
0807 11:26:57.194298 (+ 884us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:57.194313 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:57.194366 (+ 53us) server_negotiation.cc:407] Connection header received
0807 11:26:57.196214 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:57.196238 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:57.196647 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:57.197072 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:57.198620 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:57.200018 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:57.201470 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:57.202195 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:57.202420 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:57.207214 (+ 4794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:57.207222 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:57.207240 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:57.207282 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:57.210587 (+ 3305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:57.211639 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:57.211645 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:57.211652 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:57.211843 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:57.212255 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:57.212258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:57.212260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:57.212832 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:57.213057 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:57.213552 (+ 495us) server_negotiation.cc:299] Negotiation successful
0807 11:26:57.214221 (+ 669us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":393,"threads_started":1}
I20260807 11:26:57.218376 1242 raft_consensus.cc:504] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818d4fd98b34472384e0e9d8bdf31272" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46493 } }
I20260807 11:26:57.218906 1242 leader_election.cc:302] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 818d4fd98b34472384e0e9d8bdf31272; no voters:
I20260807 11:26:57.219592 1242 leader_election.cc:288] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:26:57.219729 1247 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:26:57.220460 1247 raft_consensus.cc:686] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [term 1 LEADER]: Becoming Leader. State: Replica: 818d4fd98b34472384e0e9d8bdf31272, State: Running, Role: LEADER
I20260807 11:26:57.221053 1247 consensus_queue.cc:237] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818d4fd98b34472384e0e9d8bdf31272" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46493 } }
I20260807 11:26:57.221751 1242 sys_catalog.cc:564] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:26:57.223997 1249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 818d4fd98b34472384e0e9d8bdf31272. Latest consensus state: current_term: 1 leader_uuid: "818d4fd98b34472384e0e9d8bdf31272" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818d4fd98b34472384e0e9d8bdf31272" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46493 } } }
I20260807 11:26:57.224475 1249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [sys.catalog]: This master's current role is: LEADER
I20260807 11:26:57.224817 1248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "818d4fd98b34472384e0e9d8bdf31272" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818d4fd98b34472384e0e9d8bdf31272" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46493 } } }
I20260807 11:26:57.225183 1248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [sys.catalog]: This master's current role is: LEADER
I20260807 11:26:57.226773 1254 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:26:57.232753 1254 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:26:57.242364 1254 catalog_manager.cc:1347] Generated new cluster ID: 178fafeb18fe442fbb87a63231c6d5dd
I20260807 11:26:57.242489 1254 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:26:57.257362 1254 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:26:57.259322 1254 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:26:57.284894 1254 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272: Generated new TSK 0
I20260807 11:26:57.285820 1254 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.197.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
I20260807 11:26:57.381206 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:0
--local_ip_for_outbound_sockets=127.16.197.1
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:46493
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:45231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:26:57.748284 1268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:26:57.748579 1268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:26:57.748679 1268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:26:57.756361 1268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:26:57.756494 1268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:26:57.756611 1268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:26:57.766671 1268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:45231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:46493
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:26:57.768157 1268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:26:57.769549 1268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:26:57.784097 1273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:57.784066 1274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:57.784232 1276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:57.785200 1268 server_base.cc:1034] running on GCE node
I20260807 11:26:57.785842 1268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:26:57.786733 1268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:26:57.787984 1268 hybrid_clock.cc:648] HybridClock initialized: now 1786102017787921 us; error 57 us; skew 500 ppm
I20260807 11:26:57.788316 1268 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:26:57 dist-test-slave-2w3w krb5kdc[1158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102017, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:26:57.791595 1268 init.cc:377] Logged in from keytab as kudu/127.16.197.1@KRBTEST.COM (short username kudu)
I20260807 11:26:57.793146 1268 webserver.cc:460] Webserver started at http://127.16.197.1:42255/ using document root <none> and password file <none>
I20260807 11:26:57.793844 1268 fs_manager.cc:359] Metadata directory not provided
I20260807 11:26:57.793991 1268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:26:57.794309 1268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:26:57.797374 1268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/data/instance:
uuid: "21415c113da747bc9d7db1dfd3aa073a"
format_stamp: "Formatted at 2026-08-07 11:26:57 on dist-test-slave-2w3w"
server_key: "4d911d38a98a222a513f137bce563f03"
server_key_iv: "a5f6e50f6f9f8e830ae985a847475f43"
server_key_version: "encryptionkey@0"
I20260807 11:26:57.798192 1268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance:
uuid: "21415c113da747bc9d7db1dfd3aa073a"
format_stamp: "Formatted at 2026-08-07 11:26:57 on dist-test-slave-2w3w"
server_key: "4d911d38a98a222a513f137bce563f03"
server_key_iv: "a5f6e50f6f9f8e830ae985a847475f43"
server_key_version: "encryptionkey@0"
I20260807 11:26:57.803558 1268 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 11:26:57.807072 1284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:57.808370 1268 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 11:26:57.808552 1268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "21415c113da747bc9d7db1dfd3aa073a"
format_stamp: "Formatted at 2026-08-07 11:26:57 on dist-test-slave-2w3w"
server_key: "4d911d38a98a222a513f137bce563f03"
server_key_iv: "a5f6e50f6f9f8e830ae985a847475f43"
server_key_version: "encryptionkey@0"
I20260807 11:26:57.808766 1268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:26:57.833909 1268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:26:57.838778 1268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:26:57.839161 1268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:26:57.840212 1268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:26:57.841670 1268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:26:57.841777 1268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:57.841958 1268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:26:57.842033 1268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:57.877346 1268 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:42879
I20260807 11:26:57.877406 1400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:42879 every 8 connection(s)
I20260807 11:26:57.878692 1268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:26:57.879784 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 1268
I20260807 11:26:57.880023 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance
I20260807 11:26:57.880982 17172 external_mini_cluster.cc:1428] Setting key 67bb371283a008007b153951e47c1529
Aug 07 11:26:57 dist-test-slave-2w3w krb5kdc[1158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102017, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:26:57.904932 1403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:57.881796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:58361 (local address 127.16.197.62:46493)
0807 11:26:57.882845 (+ 1049us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:57.882854 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:57.883664 (+ 810us) server_negotiation.cc:407] Connection header received
0807 11:26:57.886614 (+ 2950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:57.886618 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:57.886692 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:57.886810 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:57.889100 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:57.889859 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:57.892546 (+ 2687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:57.892991 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:57.893109 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:57.897456 (+ 4347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:57.897465 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:57.897474 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:57.897525 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:57.901252 (+ 3727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:57.902103 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:57.902111 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:57.902118 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:57.902317 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:57.902767 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:57.902774 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:57.902780 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:57.903222 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:57.903389 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:57.904161 (+ 772us) server_negotiation.cc:299] Negotiation successful
0807 11:26:57.904449 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":907,"thread_start_us":780,"threads_started":1}
I20260807 11:26:57.905576 1404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:57.882475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.1:58361)
0807 11:26:57.883474 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:57.883527 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:57.886254 (+ 2727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:57.887131 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:57.887145 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:57.888006 (+ 861us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:26:57.888875 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:57.888903 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:57.890046 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:57.890056 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:57.892349 (+ 2293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:57.892368 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:57.893252 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:57.893261 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:57.893617 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:57.894456 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:26:57.894493 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:26:57.897174 (+ 2681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:26:57.901458 (+ 4284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:57.901467 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:57.901480 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:57.901945 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:57.902449 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:57.902453 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:57.902455 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:57.902638 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:57.903355 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:26:57.903361 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:26:57.903735 (+ 374us) client_negotiation.cc:769] Sending connection context
0807 11:26:57.903981 (+ 246us) client_negotiation.cc:240] Negotiation successful
0807 11:26:57.904398 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":394,"threads_started":1}
I20260807 11:26:57.907362 1401 heartbeater.cc:347] Connected to a master server at 127.16.197.62:46493
I20260807 11:26:57.907763 1401 heartbeater.cc:464] Registering TS with master...
I20260807 11:26:57.908711 1401 heartbeater.cc:511] Master 127.16.197.62:46493 requested a full tablet report, sending...
I20260807 11:26:57.911497 1203 ts_manager.cc:194] Registered new tserver with Master: 21415c113da747bc9d7db1dfd3aa073a (127.16.197.1:42879)
I20260807 11:26:57.913338 1203 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.1@KRBTEST.COM'} at 127.16.197.1:58361
WARNING: no policy specified for kudu/127.16.197.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
I20260807 11:26:57.950141 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:0
--local_ip_for_outbound_sockets=127.16.197.2
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:46493
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:45231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:26:58.295540 1407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:26:58.295921 1407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:26:58.296113 1407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:26:58.303655 1407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:26:58.303856 1407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:26:58.304051 1407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:26:58.314589 1407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:45231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:46493
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:26:58.316167 1407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:26:58.317554 1407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:26:58.331881 1412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:58.331806 1415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:58.331781 1413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:58.332868 1407 server_base.cc:1034] running on GCE node
I20260807 11:26:58.333618 1407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:26:58.334671 1407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:26:58.335930 1407 hybrid_clock.cc:648] HybridClock initialized: now 1786102018335845 us; error 83 us; skew 500 ppm
I20260807 11:26:58.336273 1407 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:26:58 dist-test-slave-2w3w krb5kdc[1158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102018, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:26:58.339656 1407 init.cc:377] Logged in from keytab as kudu/127.16.197.2@KRBTEST.COM (short username kudu)
I20260807 11:26:58.341180 1407 webserver.cc:460] Webserver started at http://127.16.197.2:35693/ using document root <none> and password file <none>
I20260807 11:26:58.341981 1407 fs_manager.cc:359] Metadata directory not provided
I20260807 11:26:58.342139 1407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:26:58.342474 1407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:26:58.345526 1407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/data/instance:
uuid: "f1ea6fee5ffb44509a2c2d08b6400fec"
format_stamp: "Formatted at 2026-08-07 11:26:58 on dist-test-slave-2w3w"
server_key: "24230e3c61a88550440d9af17bd6e014"
server_key_iv: "3606c81b90291b46c35e81c2a7de2cde"
server_key_version: "encryptionkey@0"
I20260807 11:26:58.346316 1407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance:
uuid: "f1ea6fee5ffb44509a2c2d08b6400fec"
format_stamp: "Formatted at 2026-08-07 11:26:58 on dist-test-slave-2w3w"
server_key: "24230e3c61a88550440d9af17bd6e014"
server_key_iv: "3606c81b90291b46c35e81c2a7de2cde"
server_key_version: "encryptionkey@0"
I20260807 11:26:58.351547 1407 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 11:26:58.355047 1423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:58.356452 1407 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 11:26:58.356638 1407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "f1ea6fee5ffb44509a2c2d08b6400fec"
format_stamp: "Formatted at 2026-08-07 11:26:58 on dist-test-slave-2w3w"
server_key: "24230e3c61a88550440d9af17bd6e014"
server_key_iv: "3606c81b90291b46c35e81c2a7de2cde"
server_key_version: "encryptionkey@0"
I20260807 11:26:58.356854 1407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:26:58.408258 1407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:26:58.413074 1407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:26:58.413398 1407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:26:58.414451 1407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:26:58.415913 1407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:26:58.415989 1407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:58.416167 1407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:26:58.416218 1407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:58.451335 1407 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:45199
I20260807 11:26:58.451429 1539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:45199 every 8 connection(s)
I20260807 11:26:58.452775 1407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:26:58.457876 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 1407
I20260807 11:26:58.458076 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance
I20260807 11:26:58.458662 17172 external_mini_cluster.cc:1428] Setting key 0e0924164b82af7a6e27b0db51fcca3e
Aug 07 11:26:58 dist-test-slave-2w3w krb5kdc[1158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102018, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:26:58.475843 1542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:58.455377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:39895 (local address 127.16.197.62:46493)
0807 11:26:58.455826 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:58.455831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:58.456819 (+ 988us) server_negotiation.cc:407] Connection header received
0807 11:26:58.459347 (+ 2528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:58.459353 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:58.459453 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:58.459579 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:58.461452 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:58.462574 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:58.464220 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:58.464993 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:58.465139 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:58.468955 (+ 3816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:58.468964 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:58.468971 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:58.469020 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:58.472375 (+ 3355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:58.473403 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:58.473406 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:58.473409 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:58.473533 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:58.474030 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:58.474033 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:58.474035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:58.474290 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:58.474401 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:58.475384 (+ 983us) server_negotiation.cc:299] Negotiation successful
0807 11:26:58.475561 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":208,"threads_started":1}
I20260807 11:26:58.477451 1543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:58.455973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.2:39895)
0807 11:26:58.456656 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:58.456698 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:58.459005 (+ 2307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:58.459861 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:58.459893 (+ 32us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:58.460469 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:26:58.461249 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:58.461269 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:58.462727 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:58.462731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:58.464064 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:58.464077 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:58.465216 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:58.465221 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:58.465442 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:58.466172 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:26:58.466202 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:26:58.468702 (+ 2500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:26:58.472530 (+ 3828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:58.472542 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:58.472564 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:58.473273 (+ 709us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:58.473641 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:58.473648 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:58.473655 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:58.473917 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:58.474417 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:26:58.474428 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:26:58.474934 (+ 506us) client_negotiation.cc:769] Sending connection context
0807 11:26:58.475280 (+ 346us) client_negotiation.cc:240] Negotiation successful
0807 11:26:58.475927 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":267,"threads_started":1}
I20260807 11:26:58.479068 1540 heartbeater.cc:347] Connected to a master server at 127.16.197.62:46493
I20260807 11:26:58.479404 1540 heartbeater.cc:464] Registering TS with master...
I20260807 11:26:58.480151 1540 heartbeater.cc:511] Master 127.16.197.62:46493 requested a full tablet report, sending...
I20260807 11:26:58.481638 1203 ts_manager.cc:194] Registered new tserver with Master: f1ea6fee5ffb44509a2c2d08b6400fec (127.16.197.2:45199)
I20260807 11:26:58.482468 1203 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.2@KRBTEST.COM'} at 127.16.197.2:39895
WARNING: no policy specified for kudu/127.16.197.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
I20260807 11:26:58.523358 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:0
--local_ip_for_outbound_sockets=127.16.197.3
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:46493
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:45231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:26:58.867300 1546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:26:58.867645 1546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:26:58.867825 1546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:26:58.875433 1546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:26:58.875597 1546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:26:58.875779 1546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:26:58.886860 1546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:45231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:46493
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:26:58.888326 1546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:26:58.889693 1546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:26:58.903636 1551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:58.903537 1552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:26:58.903597 1554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:26:58.904497 1546 server_base.cc:1034] running on GCE node
I20260807 11:26:58.905079 1546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:26:58.906085 1546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:26:58.907290 1546 hybrid_clock.cc:648] HybridClock initialized: now 1786102018907240 us; error 30 us; skew 500 ppm
I20260807 11:26:58.907612 1546 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:26:58 dist-test-slave-2w3w krb5kdc[1158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102018, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:26:58.910897 1546 init.cc:377] Logged in from keytab as kudu/127.16.197.3@KRBTEST.COM (short username kudu)
I20260807 11:26:58.912551 1546 webserver.cc:460] Webserver started at http://127.16.197.3:38127/ using document root <none> and password file <none>
I20260807 11:26:58.913371 1546 fs_manager.cc:359] Metadata directory not provided
I20260807 11:26:58.913477 1546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:26:58.913806 1546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:26:58.917028 1401 heartbeater.cc:503] Master 127.16.197.62:46493 was elected leader, sending a full tablet report...
I20260807 11:26:58.917147 1546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/data/instance:
uuid: "e70ceab7bd3b4980a1a35d42a1d27d98"
format_stamp: "Formatted at 2026-08-07 11:26:58 on dist-test-slave-2w3w"
server_key: "3b15e43d3b9a3af0729c9678a90ce33c"
server_key_iv: "e15f28ab910b42670848d0fe34bdb103"
server_key_version: "encryptionkey@0"
I20260807 11:26:58.918125 1546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance:
uuid: "e70ceab7bd3b4980a1a35d42a1d27d98"
format_stamp: "Formatted at 2026-08-07 11:26:58 on dist-test-slave-2w3w"
server_key: "3b15e43d3b9a3af0729c9678a90ce33c"
server_key_iv: "e15f28ab910b42670848d0fe34bdb103"
server_key_version: "encryptionkey@0"
I20260807 11:26:58.924142 1546 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 11:26:58.928040 1561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:58.929636 1546 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 11:26:58.929872 1546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "e70ceab7bd3b4980a1a35d42a1d27d98"
format_stamp: "Formatted at 2026-08-07 11:26:58 on dist-test-slave-2w3w"
server_key: "3b15e43d3b9a3af0729c9678a90ce33c"
server_key_iv: "e15f28ab910b42670848d0fe34bdb103"
server_key_version: "encryptionkey@0"
I20260807 11:26:58.930110 1546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:26:58.958161 1546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:26:58.962910 1546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:26:58.963304 1546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:26:58.964334 1546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:26:58.965704 1546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:26:58.965780 1546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:58.965962 1546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:26:58.966017 1546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:26:58.999799 1546 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:40291
I20260807 11:26:58.999851 1677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:40291 every 8 connection(s)
I20260807 11:26:59.001178 1546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:26:59.002414 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 1546
I20260807 11:26:59.002620 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance
I20260807 11:26:59.003260 17172 external_mini_cluster.cc:1428] Setting key 113fce1711b010da58b6bc528326c916
Aug 07 11:26:59 dist-test-slave-2w3w krb5kdc[1158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102018, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:26:59.023082 1680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:59.004210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:53825 (local address 127.16.197.62:46493)
0807 11:26:59.004707 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:59.004713 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:59.005873 (+ 1160us) server_negotiation.cc:407] Connection header received
0807 11:26:59.008016 (+ 2143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:59.008021 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:59.008128 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:59.008242 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:59.009770 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.010551 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.012189 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.012602 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.012733 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:59.016270 (+ 3537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:59.016275 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:59.016279 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:59.016313 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:59.019804 (+ 3491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:59.020579 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:59.020584 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:59.020586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:59.020713 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:59.021165 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:59.021171 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:59.021179 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:59.021515 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:59.021685 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:59.022393 (+ 708us) server_negotiation.cc:299] Negotiation successful
0807 11:26:59.022627 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":225,"threads_started":1}
I20260807 11:26:59.024051 1681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:59.004843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.3:53825)
0807 11:26:59.005709 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:59.005754 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:59.007752 (+ 1998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:59.008494 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:59.008503 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:59.009031 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:26:59.009603 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:59.009620 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.010700 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.010705 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:59.012015 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:59.012062 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.012848 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.012855 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:59.013080 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:59.013760 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:26:59.013782 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:26:59.016041 (+ 2259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:26:59.019956 (+ 3915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:59.019965 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:59.019984 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:59.020439 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:59.020829 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:26:59.020834 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:26:59.020838 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:26:59.021036 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:26:59.021643 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:26:59.021652 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:26:59.022017 (+ 365us) client_negotiation.cc:769] Sending connection context
0807 11:26:59.022287 (+ 270us) client_negotiation.cc:240] Negotiation successful
0807 11:26:59.022878 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":322,"threads_started":1}
I20260807 11:26:59.025454 1678 heartbeater.cc:347] Connected to a master server at 127.16.197.62:46493
I20260807 11:26:59.025743 1678 heartbeater.cc:464] Registering TS with master...
I20260807 11:26:59.026505 1678 heartbeater.cc:511] Master 127.16.197.62:46493 requested a full tablet report, sending...
I20260807 11:26:59.028035 1203 ts_manager.cc:194] Registered new tserver with Master: e70ceab7bd3b4980a1a35d42a1d27d98 (127.16.197.3:40291)
I20260807 11:26:59.028810 1203 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.3@KRBTEST.COM'} at 127.16.197.3:53825
I20260807 11:26:59.031700 17172 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 11:26:59.051784 1680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:59.036569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42962 (local address 127.16.197.62:46493)
0807 11:26:59.036758 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:59.036765 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:59.037212 (+ 447us) server_negotiation.cc:407] Connection header received
0807 11:26:59.037440 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:59.037446 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:59.037525 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:59.037634 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:59.039224 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.039966 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.041425 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.041856 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.042007 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:59.045035 (+ 3028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:59.045056 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:59.045063 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:59.045107 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:59.048134 (+ 3027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:59.049210 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:59.049216 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:59.049220 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:59.049411 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:59.049952 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:59.049959 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:59.049963 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:59.050443 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:59.050705 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:59.051153 (+ 448us) server_negotiation.cc:299] Negotiation successful
0807 11:26:59.051416 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 11:26:59.056304 1203 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42962:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 11:26:59.063500 1203 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:26:59.092389 1694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:59.077137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:42879 (local address 127.0.0.1:49152)
0807 11:26:59.077826 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:59.077846 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:59.077981 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:59.079207 (+ 1226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:59.079215 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:59.079256 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:26:59.080450 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:59.080463 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.082927 (+ 2464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.082935 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:59.087561 (+ 4626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:59.087581 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.090738 (+ 3157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.090749 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:59.091114 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:59.091214 (+ 100us) client_negotiation.cc:769] Sending connection context
0807 11:26:59.091386 (+ 172us) client_negotiation.cc:240] Negotiation successful
0807 11:26:59.091704 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"mutex_wait_us":1754,"thread_start_us":393,"threads_started":1}
I20260807 11:26:59.092479 1692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:59.075992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:45199 (local address 127.0.0.1:45806)
0807 11:26:59.076693 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:59.076734 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:59.076929 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:59.078795 (+ 1866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:59.078804 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:59.078836 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:26:59.079422 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:59.079442 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.082296 (+ 2854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.082305 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:59.085784 (+ 3479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:59.085807 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.091158 (+ 5351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.091167 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:59.091442 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:59.091500 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 11:26:59.091590 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 11:26:59.091799 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":1,"thread_start_us":324,"threads_started":1}
I20260807 11:26:59.094323 1693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:59.076117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45806 (local address 127.16.197.2:45199)
0807 11:26:59.076695 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:59.076705 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:59.078122 (+ 1417us) server_negotiation.cc:407] Connection header received
0807 11:26:59.078235 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:59.078242 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:59.078533 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:59.079693 (+ 1160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:26:59.080012 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.082094 (+ 2082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.087206 (+ 5112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.090806 (+ 3600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.092676 (+ 1870us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:59.093001 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:59.093553 (+ 552us) server_negotiation.cc:299] Negotiation successful
0807 11:26:59.093831 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":317,"threads_started":1}
I20260807 11:26:59.095317 1695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:59.077283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49152 (local address 127.16.197.1:42879)
0807 11:26:59.078239 (+ 956us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:59.078247 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:59.078277 (+ 30us) server_negotiation.cc:407] Connection header received
0807 11:26:59.078374 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:59.078385 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:59.078648 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:59.078888 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:26:59.080670 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.082706 (+ 2036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.087835 (+ 5129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.090044 (+ 2209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.090233 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:59.090566 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:59.094702 (+ 4136us) server_negotiation.cc:299] Negotiation successful
0807 11:26:59.094927 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":781,"thread_start_us":604,"threads_started":1}
I20260807 11:26:59.098811 1473 tablet_service.cc:1467] Processing CreateTablet for tablet e26123ac006a4ecd81075b7709421742 (DEFAULT_TABLE table=test-table [id=1adcba6ec5c24f46a65ac09ed9422eab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:26:59.099020 1335 tablet_service.cc:1467] Processing CreateTablet for tablet e26123ac006a4ecd81075b7709421742 (DEFAULT_TABLE table=test-table [id=1adcba6ec5c24f46a65ac09ed9422eab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:26:59.100376 1473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e26123ac006a4ecd81075b7709421742. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:26:59.100772 1335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e26123ac006a4ecd81075b7709421742. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:26:59.102026 1692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:59.078483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:40291 (local address 127.0.0.1:54928)
0807 11:26:59.093113 (+ 14630us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:59.093141 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:59.093281 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:59.095245 (+ 1964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:59.095250 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:59.095276 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:26:59.095454 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:59.095465 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.097480 (+ 2015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.097486 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:59.099376 (+ 1890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:59.099391 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.101213 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.101219 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:59.101396 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:59.101440 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 11:26:59.101547 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 11:26:59.101713 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14482,"thread_start_us":16407,"threads_started":1}
I20260807 11:26:59.102160 1697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:59.078819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54928 (local address 127.16.197.3:40291)
0807 11:26:59.094084 (+ 15265us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:59.094094 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:59.094129 (+ 35us) server_negotiation.cc:407] Connection header received
0807 11:26:59.094234 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:59.094246 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:59.094945 (+ 699us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:59.095228 (+ 283us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:26:59.095676 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.097278 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.099581 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.101029 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.101179 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:59.101414 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:59.101609 (+ 195us) server_negotiation.cc:299] Negotiation successful
0807 11:26:59.101782 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15068,"thread_start_us":15872,"threads_started":1}
I20260807 11:26:59.106326 1611 tablet_service.cc:1467] Processing CreateTablet for tablet e26123ac006a4ecd81075b7709421742 (DEFAULT_TABLE table=test-table [id=1adcba6ec5c24f46a65ac09ed9422eab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:26:59.107972 1611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e26123ac006a4ecd81075b7709421742. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:26:59.112268 1698 tablet_bootstrap.cc:492] T e26123ac006a4ecd81075b7709421742 P f1ea6fee5ffb44509a2c2d08b6400fec: Bootstrap starting.
I20260807 11:26:59.115696 1699 tablet_bootstrap.cc:492] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a: Bootstrap starting.
I20260807 11:26:59.116365 1698 tablet_bootstrap.cc:654] T e26123ac006a4ecd81075b7709421742 P f1ea6fee5ffb44509a2c2d08b6400fec: Neither blocks nor log segments found. Creating new log.
I20260807 11:26:59.117743 1698 log.cc:824] T e26123ac006a4ecd81075b7709421742 P f1ea6fee5ffb44509a2c2d08b6400fec: Log is configured to *not* fsync() on all Append() calls
I20260807 11:26:59.120007 1699 tablet_bootstrap.cc:654] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a: Neither blocks nor log segments found. Creating new log.
I20260807 11:26:59.121209 1698 tablet_bootstrap.cc:492] T e26123ac006a4ecd81075b7709421742 P f1ea6fee5ffb44509a2c2d08b6400fec: No bootstrap required, opened a new log
I20260807 11:26:59.121425 1699 log.cc:824] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a: Log is configured to *not* fsync() on all Append() calls
I20260807 11:26:59.121501 1698 ts_tablet_manager.cc:1397] T e26123ac006a4ecd81075b7709421742 P f1ea6fee5ffb44509a2c2d08b6400fec: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260807 11:26:59.122203 1701 tablet_bootstrap.cc:492] T e26123ac006a4ecd81075b7709421742 P e70ceab7bd3b4980a1a35d42a1d27d98: Bootstrap starting.
I20260807 11:26:59.125089 1701 tablet_bootstrap.cc:654] T e26123ac006a4ecd81075b7709421742 P e70ceab7bd3b4980a1a35d42a1d27d98: Neither blocks nor log segments found. Creating new log.
I20260807 11:26:59.126423 1701 log.cc:824] T e26123ac006a4ecd81075b7709421742 P e70ceab7bd3b4980a1a35d42a1d27d98: Log is configured to *not* fsync() on all Append() calls
I20260807 11:26:59.129057 1699 tablet_bootstrap.cc:492] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a: No bootstrap required, opened a new log
I20260807 11:26:59.129374 1699 ts_tablet_manager.cc:1397] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.008s
I20260807 11:26:59.130067 1701 tablet_bootstrap.cc:492] T e26123ac006a4ecd81075b7709421742 P e70ceab7bd3b4980a1a35d42a1d27d98: No bootstrap required, opened a new log
I20260807 11:26:59.130326 1701 ts_tablet_manager.cc:1397] T e26123ac006a4ecd81075b7709421742 P e70ceab7bd3b4980a1a35d42a1d27d98: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260807 11:26:59.133162 1698 raft_consensus.cc:348] T e26123ac006a4ecd81075b7709421742 P f1ea6fee5ffb44509a2c2d08b6400fec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 45199 } } peers { permanent_uuid: "21415c113da747bc9d7db1dfd3aa073a" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 42879 } } peers { permanent_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40291 } }
I20260807 11:26:59.133673 1698 raft_consensus.cc:374] T e26123ac006a4ecd81075b7709421742 P f1ea6fee5ffb44509a2c2d08b6400fec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:26:59.133790 1698 raft_consensus.cc:729] T e26123ac006a4ecd81075b7709421742 P f1ea6fee5ffb44509a2c2d08b6400fec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1ea6fee5ffb44509a2c2d08b6400fec, State: Initialized, Role: FOLLOWER
I20260807 11:26:59.134513 1698 consensus_queue.cc:260] T e26123ac006a4ecd81075b7709421742 P f1ea6fee5ffb44509a2c2d08b6400fec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 45199 } } peers { permanent_uuid: "21415c113da747bc9d7db1dfd3aa073a" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 42879 } } peers { permanent_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40291 } }
I20260807 11:26:59.135947 1540 heartbeater.cc:503] Master 127.16.197.62:46493 was elected leader, sending a full tablet report...
I20260807 11:26:59.136570 1698 ts_tablet_manager.cc:1428] T e26123ac006a4ecd81075b7709421742 P f1ea6fee5ffb44509a2c2d08b6400fec: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260807 11:26:59.139402 1701 raft_consensus.cc:348] T e26123ac006a4ecd81075b7709421742 P e70ceab7bd3b4980a1a35d42a1d27d98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 45199 } } peers { permanent_uuid: "21415c113da747bc9d7db1dfd3aa073a" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 42879 } } peers { permanent_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40291 } }
I20260807 11:26:59.139789 1701 raft_consensus.cc:374] T e26123ac006a4ecd81075b7709421742 P e70ceab7bd3b4980a1a35d42a1d27d98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:26:59.140210 1701 raft_consensus.cc:729] T e26123ac006a4ecd81075b7709421742 P e70ceab7bd3b4980a1a35d42a1d27d98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e70ceab7bd3b4980a1a35d42a1d27d98, State: Initialized, Role: FOLLOWER
I20260807 11:26:59.140144 1699 raft_consensus.cc:348] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 45199 } } peers { permanent_uuid: "21415c113da747bc9d7db1dfd3aa073a" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 42879 } } peers { permanent_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40291 } }
I20260807 11:26:59.140588 1699 raft_consensus.cc:374] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:26:59.140702 1699 raft_consensus.cc:729] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21415c113da747bc9d7db1dfd3aa073a, State: Initialized, Role: FOLLOWER
I20260807 11:26:59.141065 1701 consensus_queue.cc:260] T e26123ac006a4ecd81075b7709421742 P e70ceab7bd3b4980a1a35d42a1d27d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 45199 } } peers { permanent_uuid: "21415c113da747bc9d7db1dfd3aa073a" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 42879 } } peers { permanent_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40291 } }
I20260807 11:26:59.141507 1699 consensus_queue.cc:260] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 45199 } } peers { permanent_uuid: "21415c113da747bc9d7db1dfd3aa073a" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 42879 } } peers { permanent_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40291 } }
I20260807 11:26:59.142553 1678 heartbeater.cc:503] Master 127.16.197.62:46493 was elected leader, sending a full tablet report...
I20260807 11:26:59.143227 1701 ts_tablet_manager.cc:1428] T e26123ac006a4ecd81075b7709421742 P e70ceab7bd3b4980a1a35d42a1d27d98: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
I20260807 11:26:59.144946 1699 ts_tablet_manager.cc:1428] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
W20260807 11:26:59.204881 1541 tablet.cc:2369] T e26123ac006a4ecd81075b7709421742 P f1ea6fee5ffb44509a2c2d08b6400fec: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 11:26:59.253096 1679 tablet.cc:2369] T e26123ac006a4ecd81075b7709421742 P e70ceab7bd3b4980a1a35d42a1d27d98: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:26:59.370348 1706 raft_consensus.cc:482] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:26:59.370658 1706 raft_consensus.cc:504] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 45199 } } peers { permanent_uuid: "21415c113da747bc9d7db1dfd3aa073a" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 42879 } } peers { permanent_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40291 } }
I20260807 11:26:59.372617 1706 leader_election.cc:288] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1ea6fee5ffb44509a2c2d08b6400fec (127.16.197.2:45199), e70ceab7bd3b4980a1a35d42a1d27d98 (127.16.197.3:40291)
I20260807 11:26:59.380316 1693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:59.373207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:47247 (local address 127.16.197.2:45199)
0807 11:26:59.373410 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:59.373418 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:59.374333 (+ 915us) server_negotiation.cc:407] Connection header received
0807 11:26:59.374510 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:59.374517 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:59.374607 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:59.374690 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:26:59.375499 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.376388 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.378368 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.379320 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.379436 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:59.379534 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:59.379891 (+ 357us) server_negotiation.cc:299] Negotiation successful
0807 11:26:59.380037 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 11:26:59.380410 1697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:59.373663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:47617 (local address 127.16.197.3:40291)
0807 11:26:59.373899 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:59.373906 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:59.374507 (+ 601us) server_negotiation.cc:407] Connection header received
0807 11:26:59.374684 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:59.374691 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:59.374781 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:59.374927 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:26:59.375886 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.376728 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.378552 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.379387 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.379549 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:59.379621 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:59.380013 (+ 392us) server_negotiation.cc:299] Negotiation successful
0807 11:26:59.380171 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 11:26:59.380475 1707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:59.373169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:40291 (local address 127.16.197.1:47617)
0807 11:26:59.374382 (+ 1213us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:59.374399 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:59.374537 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:59.375006 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:59.375014 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:59.375046 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:26:59.375662 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:59.375678 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.376897 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.376901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:59.378400 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:59.378415 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.379630 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.379635 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:59.379807 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:59.379849 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 11:26:59.379962 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 11:26:59.380075 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1079,"mutex_wait_us":154,"thread_start_us":729,"threads_started":1}
I20260807 11:26:59.381011 1708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:59.372854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:45199 (local address 127.16.197.1:47247)
0807 11:26:59.374211 (+ 1357us) negotiation.cc:107] Waiting for socket to connect
0807 11:26:59.374228 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 11:26:59.374356 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:26:59.374748 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:26:59.374753 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:26:59.374780 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:26:59.375284 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:59.375307 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.376529 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.376534 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:59.378186 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:26:59.378199 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.379536 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.379543 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:26:59.379712 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:59.379736 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 11:26:59.379812 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 11:26:59.379920 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1143,"mutex_wait_us":19,"spinlock_wait_cycles":22784,"thread_start_us":1126,"threads_started":1}
W20260807 11:26:59.381533 1402 tablet.cc:2369] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:26:59.381700 1632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e26123ac006a4ecd81075b7709421742" candidate_uuid: "21415c113da747bc9d7db1dfd3aa073a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" is_pre_election: true
I20260807 11:26:59.382179 1495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e26123ac006a4ecd81075b7709421742" candidate_uuid: "21415c113da747bc9d7db1dfd3aa073a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" is_pre_election: true
I20260807 11:26:59.382324 1632 raft_consensus.cc:2454] T e26123ac006a4ecd81075b7709421742 P e70ceab7bd3b4980a1a35d42a1d27d98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21415c113da747bc9d7db1dfd3aa073a in term 0.
I20260807 11:26:59.382730 1495 raft_consensus.cc:2454] T e26123ac006a4ecd81075b7709421742 P f1ea6fee5ffb44509a2c2d08b6400fec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21415c113da747bc9d7db1dfd3aa073a in term 0.
I20260807 11:26:59.383119 1286 leader_election.cc:302] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21415c113da747bc9d7db1dfd3aa073a, e70ceab7bd3b4980a1a35d42a1d27d98; no voters:
I20260807 11:26:59.383628 1706 raft_consensus.cc:2781] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:26:59.383819 1706 raft_consensus.cc:482] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:26:59.383944 1706 raft_consensus.cc:3037] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:26:59.386209 1706 raft_consensus.cc:504] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 45199 } } peers { permanent_uuid: "21415c113da747bc9d7db1dfd3aa073a" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 42879 } } peers { permanent_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40291 } }
I20260807 11:26:59.386898 1706 leader_election.cc:288] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a [CANDIDATE]: Term 1 election: Requested vote from peers f1ea6fee5ffb44509a2c2d08b6400fec (127.16.197.2:45199), e70ceab7bd3b4980a1a35d42a1d27d98 (127.16.197.3:40291)
I20260807 11:26:59.387410 1495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e26123ac006a4ecd81075b7709421742" candidate_uuid: "21415c113da747bc9d7db1dfd3aa073a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec"
I20260807 11:26:59.387439 1632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e26123ac006a4ecd81075b7709421742" candidate_uuid: "21415c113da747bc9d7db1dfd3aa073a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98"
I20260807 11:26:59.387609 1495 raft_consensus.cc:3037] T e26123ac006a4ecd81075b7709421742 P f1ea6fee5ffb44509a2c2d08b6400fec [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:26:59.387646 1632 raft_consensus.cc:3037] T e26123ac006a4ecd81075b7709421742 P e70ceab7bd3b4980a1a35d42a1d27d98 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:26:59.389850 1632 raft_consensus.cc:2454] T e26123ac006a4ecd81075b7709421742 P e70ceab7bd3b4980a1a35d42a1d27d98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21415c113da747bc9d7db1dfd3aa073a in term 1.
I20260807 11:26:59.389855 1495 raft_consensus.cc:2454] T e26123ac006a4ecd81075b7709421742 P f1ea6fee5ffb44509a2c2d08b6400fec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21415c113da747bc9d7db1dfd3aa073a in term 1.
I20260807 11:26:59.390283 1286 leader_election.cc:302] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21415c113da747bc9d7db1dfd3aa073a, e70ceab7bd3b4980a1a35d42a1d27d98; no voters:
I20260807 11:26:59.390591 1706 raft_consensus.cc:2781] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:26:59.391148 1706 raft_consensus.cc:686] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a [term 1 LEADER]: Becoming Leader. State: Replica: 21415c113da747bc9d7db1dfd3aa073a, State: Running, Role: LEADER
I20260807 11:26:59.391786 1706 consensus_queue.cc:237] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 45199 } } peers { permanent_uuid: "21415c113da747bc9d7db1dfd3aa073a" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 42879 } } peers { permanent_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40291 } }
I20260807 11:26:59.396178 1202 catalog_manager.cc:5495] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a reported cstate change: term changed from 0 to 1, leader changed from <none> to 21415c113da747bc9d7db1dfd3aa073a (127.16.197.1). New cstate: current_term: 1 leader_uuid: "21415c113da747bc9d7db1dfd3aa073a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 45199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21415c113da747bc9d7db1dfd3aa073a" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 42879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40291 } health_report { overall_health: UNKNOWN } } }
I20260807 11:26:59.457963 1695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:59.451762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49156 (local address 127.16.197.1:42879)
0807 11:26:59.451979 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:59.451984 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:59.452002 (+ 18us) server_negotiation.cc:407] Connection header received
0807 11:26:59.452187 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:59.452191 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:59.452279 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:59.452402 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:26:59.452864 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.453576 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.455626 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.456204 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.456302 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:59.456883 (+ 581us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:26:59.457366 (+ 483us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:26:59.457471 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:59.457595 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 11:26:59.457663 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 11:26:59.466324 1632 raft_consensus.cc:1261] T e26123ac006a4ecd81075b7709421742 P e70ceab7bd3b4980a1a35d42a1d27d98 [term 1 FOLLOWER]: Refusing update from remote peer 21415c113da747bc9d7db1dfd3aa073a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:26:59.466318 1495 raft_consensus.cc:1261] T e26123ac006a4ecd81075b7709421742 P f1ea6fee5ffb44509a2c2d08b6400fec [term 1 FOLLOWER]: Refusing update from remote peer 21415c113da747bc9d7db1dfd3aa073a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:26:59.467319 1709 consensus_queue.cc:1035] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:26:59.467724 1706 consensus_queue.cc:1035] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 45199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:26:59.487296 1714 mvcc.cc:205] Tried to move back new op lower bound from 7315873871724396544 to 7315873871432376320. Current Snapshot: MvccSnapshot[applied={T|T < 7315873871724396544}]
I20260807 11:26:59.494453 1203 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:26:59.494730 1203 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:26:59.498517 1203 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272: Sending DeleteTablet for 3 replicas of tablet e26123ac006a4ecd81075b7709421742
I20260807 11:26:59.499841 1473 tablet_service.cc:1514] Processing DeleteTablet for tablet e26123ac006a4ecd81075b7709421742 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:26:59 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:45806
I20260807 11:26:59.499958 1335 tablet_service.cc:1514] Processing DeleteTablet for tablet e26123ac006a4ecd81075b7709421742 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:26:59 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:49152
I20260807 11:26:59.500052 1611 tablet_service.cc:1514] Processing DeleteTablet for tablet e26123ac006a4ecd81075b7709421742 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:26:59 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:54928
I20260807 11:26:59.501294 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 1268
I20260807 11:26:59.512272 1723 tablet_replica.cc:330] T e26123ac006a4ecd81075b7709421742 P f1ea6fee5ffb44509a2c2d08b6400fec: stopping tablet replica
I20260807 11:26:59.512859 1723 raft_consensus.cc:2229] T e26123ac006a4ecd81075b7709421742 P f1ea6fee5ffb44509a2c2d08b6400fec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:26:59.513499 1723 raft_consensus.cc:2258] T e26123ac006a4ecd81075b7709421742 P f1ea6fee5ffb44509a2c2d08b6400fec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:26:59.514437 1722 tablet_replica.cc:330] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a: stopping tablet replica
I20260807 11:26:59.514989 1722 raft_consensus.cc:2229] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a [term 1 LEADER]: Raft consensus shutting down.
I20260807 11:26:59.515797 1722 raft_consensus.cc:2258] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:26:59.518668 1723 ts_tablet_manager.cc:1905] T e26123ac006a4ecd81075b7709421742 P f1ea6fee5ffb44509a2c2d08b6400fec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:26:59.521188 1722 ts_tablet_manager.cc:1905] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:26:59.528407 1725 tablet_replica.cc:330] T e26123ac006a4ecd81075b7709421742 P e70ceab7bd3b4980a1a35d42a1d27d98: stopping tablet replica
I20260807 11:26:59.528996 1725 raft_consensus.cc:2229] T e26123ac006a4ecd81075b7709421742 P e70ceab7bd3b4980a1a35d42a1d27d98 [term 1 FOLLOWER]: Raft consensus shutting down.
Aug 07 11:26:59 dist-test-slave-2w3w krb5kdc[1158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.1@KRBTEST.COM
I20260807 11:26:59.532086 1725 raft_consensus.cc:2258] T e26123ac006a4ecd81075b7709421742 P e70ceab7bd3b4980a1a35d42a1d27d98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:26:59.532366 1723 ts_tablet_manager.cc:1918] T e26123ac006a4ecd81075b7709421742 P f1ea6fee5ffb44509a2c2d08b6400fec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:26:59.532546 1723 log.cc:1196] T e26123ac006a4ecd81075b7709421742 P f1ea6fee5ffb44509a2c2d08b6400fec: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/wal/wals/e26123ac006a4ecd81075b7709421742
I20260807 11:26:59.532950 1722 ts_tablet_manager.cc:1918] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:26:59.533113 1723 ts_tablet_manager.cc:1939] T e26123ac006a4ecd81075b7709421742 P f1ea6fee5ffb44509a2c2d08b6400fec: Deleting consensus metadata
I20260807 11:26:59.533143 1722 log.cc:1196] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/wal/wals/e26123ac006a4ecd81075b7709421742
I20260807 11:26:59.533783 1722 ts_tablet_manager.cc:1939] T e26123ac006a4ecd81075b7709421742 P 21415c113da747bc9d7db1dfd3aa073a: Deleting consensus metadata
I20260807 11:26:59.535368 1187 catalog_manager.cc:4841] TS f1ea6fee5ffb44509a2c2d08b6400fec (127.16.197.2:45199): tablet e26123ac006a4ecd81075b7709421742 (table test-table [id=1adcba6ec5c24f46a65ac09ed9422eab]) successfully deleted
I20260807 11:26:59.535931 1190 catalog_manager.cc:4841] TS 21415c113da747bc9d7db1dfd3aa073a (127.16.197.1:42879): tablet e26123ac006a4ecd81075b7709421742 (table test-table [id=1adcba6ec5c24f46a65ac09ed9422eab]) successfully deleted
I20260807 11:26:59.536352 1725 ts_tablet_manager.cc:1905] T e26123ac006a4ecd81075b7709421742 P e70ceab7bd3b4980a1a35d42a1d27d98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:26:59.543546 1725 ts_tablet_manager.cc:1918] T e26123ac006a4ecd81075b7709421742 P e70ceab7bd3b4980a1a35d42a1d27d98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:26:59.543627 1695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:59.502867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49158 (local address 127.16.197.1:42879)
0807 11:26:59.503134 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:59.503143 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:59.518114 (+ 14971us) server_negotiation.cc:407] Connection header received
0807 11:26:59.518636 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:59.518642 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:59.518734 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:59.518829 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:59.521190 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.522383 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.524556 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.525286 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.525397 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:59.537024 (+ 11627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:59.537036 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:59.537050 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:59.537093 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:59.540344 (+ 3251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:59.541164 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:59.541171 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:59.541174 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:59.541327 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:59.541765 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:59.541769 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:59.541771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:59.542136 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:59.542481 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:59.542916 (+ 435us) server_negotiation.cc:299] Negotiation successful
0807 11:26:59.543213 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 11:26:59.543730 1725 log.cc:1196] T e26123ac006a4ecd81075b7709421742 P e70ceab7bd3b4980a1a35d42a1d27d98: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/wal/wals/e26123ac006a4ecd81075b7709421742
I20260807 11:26:59.544382 1725 ts_tablet_manager.cc:1939] T e26123ac006a4ecd81075b7709421742 P e70ceab7bd3b4980a1a35d42a1d27d98: Deleting consensus metadata
I20260807 11:26:59.544402 1396 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:26:59.546370 1188 catalog_manager.cc:4841] TS e70ceab7bd3b4980a1a35d42a1d27d98 (127.16.197.3:40291): tablet e26123ac006a4ecd81075b7709421742 (table test-table [id=1adcba6ec5c24f46a65ac09ed9422eab]) successfully deleted
I20260807 11:26:59.650611 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 1268
I20260807 11:26:59.672719 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 1407
Aug 07 11:26:59 dist-test-slave-2w3w krb5kdc[1158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.2@KRBTEST.COM
I20260807 11:26:59.689110 1693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:59.673581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45808 (local address 127.16.197.2:45199)
0807 11:26:59.673752 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:59.673756 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:59.673776 (+ 20us) server_negotiation.cc:407] Connection header received
0807 11:26:59.673930 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:59.673935 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:59.674001 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:59.674092 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:59.675525 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.676249 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.677563 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.678020 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.678115 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:59.683515 (+ 5400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:59.683532 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:59.683548 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:59.683585 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:59.686410 (+ 2825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:59.687230 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:59.687238 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:59.687240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:59.687395 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:59.687742 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:59.687745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:59.687747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:59.688140 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:59.688388 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:59.688660 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 11:26:59.688837 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 11:26:59.689815 1535 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:26:59.781978 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 1407
I20260807 11:26:59.801431 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 1546
Aug 07 11:26:59 dist-test-slave-2w3w krb5kdc[1158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.3@KRBTEST.COM
I20260807 11:26:59.818084 1697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:26:59.802254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54940 (local address 127.16.197.3:40291)
0807 11:26:59.802428 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 11:26:59.802433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:26:59.802472 (+ 39us) server_negotiation.cc:407] Connection header received
0807 11:26:59.802659 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:26:59.802664 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:26:59.802742 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:26:59.802829 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:26:59.804309 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.805037 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.806393 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:26:59.806775 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:26:59.807202 (+ 427us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:26:59.811834 (+ 4632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:26:59.811852 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:26:59.811872 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:26:59.811908 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:26:59.814890 (+ 2982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:59.815925 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:59.815933 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:59.815935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:59.816103 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:26:59.816495 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:26:59.816499 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:26:59.816501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:26:59.816887 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:26:59.817116 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0807 11:26:59.817583 (+ 467us) server_negotiation.cc:299] Negotiation successful
0807 11:26:59.817786 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 11:26:59.818761 1673 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:26:59.909725 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 1546
I20260807 11:26:59.929556 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 1171
I20260807 11:26:59.930784 1237 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:27:00.044386 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 1171
I20260807 11:27:20.069109 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:46493
--webserver_interface=127.16.197.62
--webserver_port=43227
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:45231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:46493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:27:20.408953 1731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:27:20.409238 1731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:27:20.409417 1731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:27:20.416782 1731 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 11:27:20.416889 1731 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 11:27:20.416976 1731 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 11:27:20.417083 1731 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:27:20.417176 1731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:27:20.417297 1731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:27:20.417377 1731 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:27:20.417469 1731 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:27:20.427450 1731 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:45231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:46493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:46493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=43227
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:27:20.428762 1731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:27:20.430151 1731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:27:20.455521 1737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:27:20.455461 1738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:27:20.455498 1740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:27:20.460485 1731 server_base.cc:1034] running on GCE node
I20260807 11:27:20.461366 1731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:27:20.463008 1731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:27:20.464278 1731 hybrid_clock.cc:648] HybridClock initialized: now 1786102040464200 us; error 55 us; skew 500 ppm
I20260807 11:27:20.464684 1731 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:27:20 dist-test-slave-2w3w krb5kdc[1158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102040, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:27:20.471979 1731 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:27:20.473760 1731 webserver.cc:460] Webserver started at http://127.16.197.62:43227/ using document root <none> and password file <none>
I20260807 11:27:20.474539 1731 fs_manager.cc:359] Metadata directory not provided
I20260807 11:27:20.474645 1731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:27:20.480783 1731 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 11:27:20.483770 1747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:27:20.485167 1731 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:27:20.485342 1731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "818d4fd98b34472384e0e9d8bdf31272"
format_stamp: "Formatted at 2026-08-07 11:26:57 on dist-test-slave-2w3w"
server_key: "018456d574668f87b119ebe7c05d051e"
server_key_iv: "a27613eb1c58dac2ebe84e6d2e3ae5e9"
server_key_version: "encryptionkey@0"
I20260807 11:27:20.486450 1731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:27:20.501925 1731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:27:20.506147 1731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:27:20.506474 1731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:27:20.526046 1731 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:46493
I20260807 11:27:20.526070 1799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:46493 every 8 connection(s)
I20260807 11:27:20.527635 1731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:27:20.534803 1800 sys_catalog.cc:263] Verifying existing consensus state
I20260807 11:27:20.537565 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 1731
I20260807 11:27:20.538640 1800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272: Bootstrap starting.
I20260807 11:27:20.538653 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:42879
--local_ip_for_outbound_sockets=127.16.197.1
--tserver_master_addrs=127.16.197.62:46493
--webserver_port=42255
--webserver_interface=127.16.197.1
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:45231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 11:27:20.562479 1800 log.cc:824] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272: Log is configured to *not* fsync() on all Append() calls
I20260807 11:27:20.575196 1800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 11:27:20.575742 1800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272: Bootstrap complete.
I20260807 11:27:20.585458 1800 raft_consensus.cc:348] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818d4fd98b34472384e0e9d8bdf31272" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46493 } }
I20260807 11:27:20.586346 1800 raft_consensus.cc:729] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 818d4fd98b34472384e0e9d8bdf31272, State: Initialized, Role: FOLLOWER
I20260807 11:27:20.587085 1800 consensus_queue.cc:260] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818d4fd98b34472384e0e9d8bdf31272" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46493 } }
I20260807 11:27:20.587333 1800 raft_consensus.cc:388] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:27:20.587467 1800 raft_consensus.cc:482] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:27:20.587663 1800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [term 1 FOLLOWER]: Advancing to term 2
I20260807 11:27:20.592069 1800 raft_consensus.cc:504] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818d4fd98b34472384e0e9d8bdf31272" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46493 } }
I20260807 11:27:20.592640 1800 leader_election.cc:302] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 818d4fd98b34472384e0e9d8bdf31272; no voters:
I20260807 11:27:20.593317 1800 leader_election.cc:288] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 11:27:20.593479 1804 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 11:27:20.596000 1800 sys_catalog.cc:564] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:27:20.600616 1804 raft_consensus.cc:686] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [term 2 LEADER]: Becoming Leader. State: Replica: 818d4fd98b34472384e0e9d8bdf31272, State: Running, Role: LEADER
I20260807 11:27:20.601413 1804 consensus_queue.cc:237] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818d4fd98b34472384e0e9d8bdf31272" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46493 } }
I20260807 11:27:20.604620 1805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "818d4fd98b34472384e0e9d8bdf31272" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818d4fd98b34472384e0e9d8bdf31272" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46493 } } }
I20260807 11:27:20.605139 1805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [sys.catalog]: This master's current role is: LEADER
I20260807 11:27:20.605908 1806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 818d4fd98b34472384e0e9d8bdf31272. Latest consensus state: current_term: 2 leader_uuid: "818d4fd98b34472384e0e9d8bdf31272" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818d4fd98b34472384e0e9d8bdf31272" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46493 } } }
I20260807 11:27:20.606259 1806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [sys.catalog]: This master's current role is: LEADER
I20260807 11:27:20.608307 1816 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:27:20.620057 1816 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:27:20.622072 1816 catalog_manager.cc:1259] Loaded cluster ID: 178fafeb18fe442fbb87a63231c6d5dd
I20260807 11:27:20.622288 1816 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:27:20.627774 1816 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:27:20.630014 1816 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272: Loaded TSK: 0
I20260807 11:27:20.663388 1816 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272: Generated new TSK 1
I20260807 11:27:20.663839 1816 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 11:27:20.945539 1802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:27:20.945894 1802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:27:20.946168 1802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:27:20.954125 1802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:27:20.954288 1802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:27:20.954478 1802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:27:20.964789 1802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:45231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:42879
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=42255
--tserver_master_addrs=127.16.197.62:46493
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:27:20.966493 1802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:27:20.967916 1802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:27:20.982033 1829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:27:20.982306 1828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:27:20.982776 1831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:27:20.983305 1802 server_base.cc:1034] running on GCE node
I20260807 11:27:20.984048 1802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:27:20.985148 1802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:27:20.986413 1802 hybrid_clock.cc:648] HybridClock initialized: now 1786102040986331 us; error 60 us; skew 500 ppm
I20260807 11:27:20.986757 1802 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:27:20 dist-test-slave-2w3w krb5kdc[1158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102040, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:27:20.990183 1802 init.cc:377] Logged in from keytab as kudu/127.16.197.1@KRBTEST.COM (short username kudu)
I20260807 11:27:20.991928 1802 webserver.cc:460] Webserver started at http://127.16.197.1:42255/ using document root <none> and password file <none>
I20260807 11:27:20.992648 1802 fs_manager.cc:359] Metadata directory not provided
I20260807 11:27:20.992801 1802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:27:20.999096 1802 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 11:27:21.002040 1838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:27:21.003778 1802 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:27:21.004007 1802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "21415c113da747bc9d7db1dfd3aa073a"
format_stamp: "Formatted at 2026-08-07 11:26:57 on dist-test-slave-2w3w"
server_key: "4d911d38a98a222a513f137bce563f03"
server_key_iv: "a5f6e50f6f9f8e830ae985a847475f43"
server_key_version: "encryptionkey@0"
I20260807 11:27:21.005146 1802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:27:21.030601 1802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:27:21.035218 1802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:27:21.035522 1802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:27:21.036809 1802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:27:21.038331 1802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:27:21.038470 1802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:27:21.038611 1802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:27:21.038705 1802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:27:21.076653 1802 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:42879
I20260807 11:27:21.076740 1957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:42879 every 8 connection(s)
I20260807 11:27:21.078181 1802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:27:21.080871 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 1802
I20260807 11:27:21.081756 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:45199
--local_ip_for_outbound_sockets=127.16.197.2
--tserver_master_addrs=127.16.197.62:46493
--webserver_port=35693
--webserver_interface=127.16.197.2
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:45231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 11:27:21 dist-test-slave-2w3w krb5kdc[1158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102040, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:27:21.107191 1961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:21.081495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.1:43201)
0807 11:27:21.083596 (+ 2101us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:21.083644 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 11:27:21.085468 (+ 1824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:27:21.087270 (+ 1802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:27:21.087277 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:27:21.087822 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:27:21.088365 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:27:21.088381 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:27:21.090915 (+ 2534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:27:21.090919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:27:21.093066 (+ 2147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:27:21.093085 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:27:21.094184 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:27:21.094194 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:27:21.094523 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:27:21.095415 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:27:21.095452 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:27:21.098444 (+ 2992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:27:21.102158 (+ 3714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:27:21.102168 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:27:21.102184 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:27:21.102697 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:27:21.103292 (+ 595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:27:21.103297 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:27:21.103301 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:27:21.103561 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:27:21.104538 (+ 977us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:27:21.104547 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:27:21.104899 (+ 352us) client_negotiation.cc:769] Sending connection context
0807 11:27:21.105173 (+ 274us) client_negotiation.cc:240] Negotiation successful
0807 11:27:21.105777 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1808,"thread_start_us":1557,"threads_started":1}
I20260807 11:27:21.107188 1960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:21.081363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:43201 (local address 127.16.197.62:46493)
0807 11:27:21.082140 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0807 11:27:21.082162 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 11:27:21.083823 (+ 1661us) server_negotiation.cc:407] Connection header received
0807 11:27:21.086093 (+ 2270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:27:21.086115 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:27:21.086611 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:27:21.087119 (+ 508us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:27:21.088620 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:21.090722 (+ 2102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:21.093249 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:21.093923 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:21.094081 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:27:21.098758 (+ 4677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:27:21.098766 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:27:21.098788 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:27:21.098828 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:27:21.101968 (+ 3140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:27:21.102866 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:27:21.102873 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:27:21.102879 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:27:21.103153 (+ 274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:27:21.103687 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:27:21.103693 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:27:21.103696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:27:21.104383 (+ 687us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:27:21.104582 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 11:27:21.105416 (+ 834us) server_negotiation.cc:299] Negotiation successful
0807 11:27:21.105812 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":257,"threads_started":1}
I20260807 11:27:21.110916 1958 heartbeater.cc:347] Connected to a master server at 127.16.197.62:46493
I20260807 11:27:21.111280 1958 heartbeater.cc:464] Registering TS with master...
I20260807 11:27:21.112002 1958 heartbeater.cc:511] Master 127.16.197.62:46493 requested a full tablet report, sending...
I20260807 11:27:21.114292 1764 ts_manager.cc:194] Registered new tserver with Master: 21415c113da747bc9d7db1dfd3aa073a (127.16.197.1:42879)
I20260807 11:27:21.115803 1764 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.1@KRBTEST.COM'} at 127.16.197.1:43201
W20260807 11:27:21.425534 1964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:27:21.425819 1964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:27:21.425978 1964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:27:21.434110 1964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:27:21.434226 1964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:27:21.434406 1964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:27:21.444682 1964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:45231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:45199
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=35693
--tserver_master_addrs=127.16.197.62:46493
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:27:21.446012 1964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:27:21.447387 1964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:27:21.460676 1972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:27:21.460676 1969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:27:21.460764 1970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:27:21.461512 1964 server_base.cc:1034] running on GCE node
I20260807 11:27:21.462193 1964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:27:21.463176 1964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:27:21.464401 1964 hybrid_clock.cc:648] HybridClock initialized: now 1786102041464359 us; error 36 us; skew 500 ppm
I20260807 11:27:21.464715 1964 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:27:21 dist-test-slave-2w3w krb5kdc[1158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102041, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:27:21.467821 1964 init.cc:377] Logged in from keytab as kudu/127.16.197.2@KRBTEST.COM (short username kudu)
I20260807 11:27:21.469218 1964 webserver.cc:460] Webserver started at http://127.16.197.2:35693/ using document root <none> and password file <none>
I20260807 11:27:21.469926 1964 fs_manager.cc:359] Metadata directory not provided
I20260807 11:27:21.470095 1964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:27:21.475698 1964 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 11:27:21.478269 1979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:27:21.479637 1964 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 11:27:21.479815 1964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "f1ea6fee5ffb44509a2c2d08b6400fec"
format_stamp: "Formatted at 2026-08-07 11:26:58 on dist-test-slave-2w3w"
server_key: "24230e3c61a88550440d9af17bd6e014"
server_key_iv: "3606c81b90291b46c35e81c2a7de2cde"
server_key_version: "encryptionkey@0"
I20260807 11:27:21.480787 1964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:27:21.500778 1964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:27:21.505298 1964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:27:21.505635 1964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:27:21.506685 1964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:27:21.508211 1964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:27:21.508343 1964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:27:21.508479 1964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:27:21.508579 1964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:27:21.544030 1964 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:45199
I20260807 11:27:21.544090 2097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:45199 every 8 connection(s)
I20260807 11:27:21.545821 1964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:27:21.555151 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 1964
I20260807 11:27:21.555935 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:40291
--local_ip_for_outbound_sockets=127.16.197.3
--tserver_master_addrs=127.16.197.62:46493
--webserver_port=38127
--webserver_interface=127.16.197.3
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:45231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 11:27:21 dist-test-slave-2w3w krb5kdc[1158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102041, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:27:21.567869 1960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:21.548579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:52177 (local address 127.16.197.62:46493)
0807 11:27:21.548762 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 11:27:21.548767 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:27:21.550081 (+ 1314us) server_negotiation.cc:407] Connection header received
0807 11:27:21.551973 (+ 1892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:27:21.551980 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:27:21.552089 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:27:21.552202 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:27:21.553664 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:21.554377 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:21.556775 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:21.557192 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:21.557313 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:27:21.561765 (+ 4452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:27:21.561770 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:27:21.561774 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:27:21.561806 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:27:21.564441 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:27:21.565377 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:27:21.565382 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:27:21.565384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:27:21.565504 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:27:21.566041 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:27:21.566052 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:27:21.566054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:27:21.566359 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:27:21.566525 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 11:27:21.567277 (+ 752us) server_negotiation.cc:299] Negotiation successful
0807 11:27:21.567525 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 11:27:21.568841 2100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:21.549161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.2:52177)
0807 11:27:21.549919 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:21.549964 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 11:27:21.551693 (+ 1729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:27:21.552433 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:27:21.552440 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:27:21.552955 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:27:21.553492 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:27:21.553509 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:27:21.554544 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:27:21.554548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:27:21.556621 (+ 2073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:27:21.556640 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:27:21.557453 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:27:21.557463 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:27:21.557784 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:27:21.558504 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:27:21.558543 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:27:21.561529 (+ 2986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:27:21.564582 (+ 3053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:27:21.564593 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:27:21.564612 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:27:21.565245 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:27:21.565617 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:27:21.565624 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:27:21.565630 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:27:21.565894 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:27:21.566494 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:27:21.566502 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:27:21.566854 (+ 352us) client_negotiation.cc:769] Sending connection context
0807 11:27:21.567172 (+ 318us) client_negotiation.cc:240] Negotiation successful
0807 11:27:21.567707 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":304,"threads_started":1}
I20260807 11:27:21.570551 2098 heartbeater.cc:347] Connected to a master server at 127.16.197.62:46493
I20260807 11:27:21.570856 2098 heartbeater.cc:464] Registering TS with master...
I20260807 11:27:21.571601 2098 heartbeater.cc:511] Master 127.16.197.62:46493 requested a full tablet report, sending...
I20260807 11:27:21.573243 1764 ts_manager.cc:194] Registered new tserver with Master: f1ea6fee5ffb44509a2c2d08b6400fec (127.16.197.2:45199)
I20260807 11:27:21.574679 1764 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.2@KRBTEST.COM'} at 127.16.197.2:52177
W20260807 11:27:21.907233 2101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:27:21.907543 2101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:27:21.907642 2101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:27:21.915273 2101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:27:21.915393 2101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:27:21.915508 2101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:27:21.925685 2101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:45231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:40291
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=38127
--tserver_master_addrs=127.16.197.62:46493
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:27:21.927179 2101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:27:21.928478 2101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:27:21.942127 2106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:27:21.942274 2109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:27:21.942081 2107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:27:21.943665 2101 server_base.cc:1034] running on GCE node
I20260807 11:27:21.944301 2101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:27:21.945318 2101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:27:21.946560 2101 hybrid_clock.cc:648] HybridClock initialized: now 1786102041946516 us; error 44 us; skew 500 ppm
I20260807 11:27:21.946923 2101 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:27:21 dist-test-slave-2w3w krb5kdc[1158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102041, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:27:21.950234 2101 init.cc:377] Logged in from keytab as kudu/127.16.197.3@KRBTEST.COM (short username kudu)
I20260807 11:27:21.951848 2101 webserver.cc:460] Webserver started at http://127.16.197.3:38127/ using document root <none> and password file <none>
I20260807 11:27:21.952574 2101 fs_manager.cc:359] Metadata directory not provided
I20260807 11:27:21.952744 2101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:27:21.958376 2101 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 11:27:21.961217 2116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:27:21.962623 2101 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 11:27:21.962816 2101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "e70ceab7bd3b4980a1a35d42a1d27d98"
format_stamp: "Formatted at 2026-08-07 11:26:58 on dist-test-slave-2w3w"
server_key: "3b15e43d3b9a3af0729c9678a90ce33c"
server_key_iv: "e15f28ab910b42670848d0fe34bdb103"
server_key_version: "encryptionkey@0"
I20260807 11:27:21.963949 2101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:27:21.999174 2101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:27:22.004441 2101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:27:22.004781 2101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:27:22.005959 2101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:27:22.007663 2101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:27:22.007805 2101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:27:22.007958 2101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:27:22.008062 2101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:27:22.045264 2101 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:40291
I20260807 11:27:22.045325 2234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:40291 every 8 connection(s)
I20260807 11:27:22.046850 2101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:27:22.049212 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 2101
Aug 07 11:27:22 dist-test-slave-2w3w krb5kdc[1158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102041, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:27:22.068804 2239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:22.050608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40108 (local address 127.16.197.62:46493)
0807 11:27:22.054085 (+ 3477us) server_negotiation.cc:206] Beginning negotiation
0807 11:27:22.054091 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:27:22.054109 (+ 18us) server_negotiation.cc:407] Connection header received
0807 11:27:22.054160 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:27:22.054165 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:27:22.054253 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:27:22.054379 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:27:22.056242 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.057030 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.058488 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.059152 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.059286 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:27:22.062407 (+ 3121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:27:22.062413 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:27:22.062416 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:27:22.062450 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:27:22.065699 (+ 3249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:27:22.066452 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:27:22.066459 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:27:22.066463 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:27:22.066591 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:27:22.067022 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:27:22.067027 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:27:22.067031 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:27:22.067414 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:27:22.067683 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0807 11:27:22.068131 (+ 448us) server_negotiation.cc:299] Negotiation successful
0807 11:27:22.068365 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":3344,"spinlock_wait_cycles":21888,"thread_start_us":3194,"threads_started":1}
I20260807 11:27:22.069906 1960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:22.049949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:36371 (local address 127.16.197.62:46493)
0807 11:27:22.050173 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 11:27:22.050180 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:27:22.051605 (+ 1425us) server_negotiation.cc:407] Connection header received
0807 11:27:22.053717 (+ 2112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:27:22.053726 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:27:22.053836 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:27:22.053952 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:27:22.055796 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.056645 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.058383 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.059111 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.059265 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:27:22.063108 (+ 3843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:27:22.063113 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:27:22.063116 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:27:22.063149 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:27:22.066145 (+ 2996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:27:22.066996 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:27:22.067006 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:27:22.067010 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:27:22.067153 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:27:22.067747 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:27:22.067752 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:27:22.067756 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:27:22.068094 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:27:22.068212 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 11:27:22.069219 (+ 1007us) server_negotiation.cc:299] Negotiation successful
0807 11:27:22.069446 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":80,"server-negotiator.queue_time_us":89}
I20260807 11:27:22.071137 2238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:22.050469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.3:36371)
0807 11:27:22.051399 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:22.051454 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 11:27:22.053434 (+ 1980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:27:22.054263 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:27:22.054275 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:27:22.054954 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:27:22.055609 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:27:22.055628 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.056850 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.056854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:27:22.058214 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:27:22.058228 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.059391 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.059399 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:27:22.059677 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:27:22.060489 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:27:22.060513 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:27:22.062846 (+ 2333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:27:22.066302 (+ 3456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:27:22.066312 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:27:22.066330 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:27:22.066797 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:27:22.067310 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:27:22.067316 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:27:22.067319 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:27:22.067583 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:27:22.068323 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:27:22.068332 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:27:22.068771 (+ 439us) client_negotiation.cc:769] Sending connection context
0807 11:27:22.069093 (+ 322us) client_negotiation.cc:240] Negotiation successful
0807 11:27:22.069661 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":412,"threads_started":1}
I20260807 11:27:22.072607 2235 heartbeater.cc:347] Connected to a master server at 127.16.197.62:46493
I20260807 11:27:22.072919 2235 heartbeater.cc:464] Registering TS with master...
I20260807 11:27:22.073621 2235 heartbeater.cc:511] Master 127.16.197.62:46493 requested a full tablet report, sending...
I20260807 11:27:22.075075 1764 ts_manager.cc:194] Registered new tserver with Master: e70ceab7bd3b4980a1a35d42a1d27d98 (127.16.197.3:40291)
I20260807 11:27:22.076045 1764 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.3@KRBTEST.COM'} at 127.16.197.3:36371
I20260807 11:27:22.084549 17172 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 11:27:22.092711 1960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:22.085823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40112 (local address 127.16.197.62:46493)
0807 11:27:22.086003 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 11:27:22.086008 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:27:22.086452 (+ 444us) server_negotiation.cc:407] Connection header received
0807 11:27:22.086645 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:27:22.086648 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:27:22.086807 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:27:22.086935 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:27:22.087482 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.088311 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.090592 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.091236 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.091341 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:27:22.091891 (+ 550us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:27:22.092162 (+ 271us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 11:27:22.092475 (+ 313us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40112: token expired
Metrics: {"server-negotiator.queue_time_us":70}
W20260807 11:27:22.092875 1960 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40112: token expired
W20260807 11:27:22.093464 1685 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.16.197.62:46493)
I20260807 11:27:22.107295 1960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:22.093975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40120 (local address 127.16.197.62:46493)
0807 11:27:22.094162 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 11:27:22.094165 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 11:27:22.094181 (+ 16us) server_negotiation.cc:407] Connection header received
0807 11:27:22.094291 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:27:22.094294 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:27:22.094356 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:27:22.094433 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:27:22.096018 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.096833 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.098479 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.098929 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.099109 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:27:22.101903 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:27:22.101908 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:27:22.101911 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:27:22.101948 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:27:22.104707 (+ 2759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:27:22.105448 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:27:22.105452 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:27:22.105454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:27:22.105581 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:27:22.105947 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:27:22.105950 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:27:22.105952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:27:22.106267 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:27:22.106466 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 11:27:22.106819 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 11:27:22.107023 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 11:27:22.111295 1764 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40120:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 11:27:22.118597 1764 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:27:22.119053 1958 heartbeater.cc:503] Master 127.16.197.62:46493 was elected leader, sending a full tablet report...
I20260807 11:27:22.155948 2247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:22.131167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41716 (local address 127.16.197.2:45199)
0807 11:27:22.144097 (+ 12930us) server_negotiation.cc:206] Beginning negotiation
0807 11:27:22.144112 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 11:27:22.146026 (+ 1914us) server_negotiation.cc:407] Connection header received
0807 11:27:22.146559 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:27:22.146568 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:27:22.146863 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:27:22.147099 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:27:22.148310 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.149995 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.152969 (+ 2974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.154290 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.154433 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:27:22.154653 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 11:27:22.155384 (+ 731us) server_negotiation.cc:299] Negotiation successful
0807 11:27:22.155571 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12650,"thread_start_us":12957,"threads_started":1}
I20260807 11:27:22.156551 2248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:22.131067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:42879 (local address 127.0.0.1:35224)
0807 11:27:22.147252 (+ 16185us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:22.147278 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 11:27:22.147501 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:27:22.148488 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:27:22.148492 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:27:22.148520 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:27:22.149044 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:27:22.149057 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.150898 (+ 1841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.150903 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:27:22.153327 (+ 2424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:27:22.153342 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.155542 (+ 2200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.155549 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:27:22.155717 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:27:22.155751 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 11:27:22.155847 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 11:27:22.156070 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16016,"mutex_wait_us":34,"thread_start_us":17781,"threads_started":1}
I20260807 11:27:22.156934 2250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:22.132248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45082 (local address 127.16.197.3:40291)
0807 11:27:22.146674 (+ 14426us) server_negotiation.cc:206] Beginning negotiation
0807 11:27:22.146683 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:27:22.149333 (+ 2650us) server_negotiation.cc:407] Connection header received
0807 11:27:22.149489 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:27:22.149496 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:27:22.149689 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:27:22.149856 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:27:22.150489 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.152126 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.154198 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.155644 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.155814 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:27:22.156031 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 11:27:22.156391 (+ 360us) server_negotiation.cc:299] Negotiation successful
0807 11:27:22.156568 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14250,"thread_start_us":14073,"threads_started":1}
I20260807 11:27:22.156911 2246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:22.131528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:40291 (local address 127.0.0.1:45082)
0807 11:27:22.149186 (+ 17658us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:22.149205 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 11:27:22.149342 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:27:22.149883 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:27:22.149888 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:27:22.149913 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:27:22.150268 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:27:22.150284 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.152261 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.152270 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:27:22.154038 (+ 1768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:27:22.154050 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.155881 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.155887 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:27:22.156091 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:27:22.156120 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 11:27:22.156205 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 11:27:22.156371 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17496,"mutex_wait_us":3,"thread_start_us":15369,"threads_started":1}
I20260807 11:27:22.156984 2245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:22.131025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:45199 (local address 127.0.0.1:41716)
0807 11:27:22.145822 (+ 14797us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:22.145877 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 11:27:22.146224 (+ 347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:27:22.147480 (+ 1256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:27:22.147489 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:27:22.147515 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:27:22.148085 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:27:22.148104 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.150163 (+ 2059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.150172 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:27:22.152798 (+ 2626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:27:22.152810 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.154848 (+ 2038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.154855 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:27:22.155100 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:27:22.155166 (+ 66us) client_negotiation.cc:769] Sending connection context
0807 11:27:22.156252 (+ 1086us) client_negotiation.cc:240] Negotiation successful
0807 11:27:22.156441 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14524,"mutex_wait_us":133,"thread_start_us":14336,"threads_started":1}
I20260807 11:27:22.157474 2249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:22.131739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35224 (local address 127.16.197.1:42879)
0807 11:27:22.145096 (+ 13357us) server_negotiation.cc:206] Beginning negotiation
0807 11:27:22.145108 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 11:27:22.147394 (+ 2286us) server_negotiation.cc:407] Connection header received
0807 11:27:22.147743 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:27:22.147751 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:27:22.147998 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:27:22.148184 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:27:22.149235 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.150698 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.153508 (+ 2810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.155029 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.155161 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:27:22.155404 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0807 11:27:22.156917 (+ 1513us) server_negotiation.cc:299] Negotiation successful
0807 11:27:22.157102 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13118,"thread_start_us":12913,"threads_started":1}
I20260807 11:27:22.160955 2029 tablet_service.cc:1467] Processing CreateTablet for tablet 197a562dcf18493a94ced106edf9e8c5 (DEFAULT_TABLE table=test-table [id=31ab465f3c3c4000bdd2fe8faf6e4afc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:27:22.161178 1888 tablet_service.cc:1467] Processing CreateTablet for tablet 197a562dcf18493a94ced106edf9e8c5 (DEFAULT_TABLE table=test-table [id=31ab465f3c3c4000bdd2fe8faf6e4afc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:27:22.161178 2165 tablet_service.cc:1467] Processing CreateTablet for tablet 197a562dcf18493a94ced106edf9e8c5 (DEFAULT_TABLE table=test-table [id=31ab465f3c3c4000bdd2fe8faf6e4afc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:27:22.162293 2029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 197a562dcf18493a94ced106edf9e8c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:27:22.162549 2165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 197a562dcf18493a94ced106edf9e8c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:27:22.162566 1888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 197a562dcf18493a94ced106edf9e8c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:27:22.175447 2251 tablet_bootstrap.cc:492] T 197a562dcf18493a94ced106edf9e8c5 P f1ea6fee5ffb44509a2c2d08b6400fec: Bootstrap starting.
I20260807 11:27:22.178627 2252 tablet_bootstrap.cc:492] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a: Bootstrap starting.
I20260807 11:27:22.179159 2251 tablet_bootstrap.cc:654] T 197a562dcf18493a94ced106edf9e8c5 P f1ea6fee5ffb44509a2c2d08b6400fec: Neither blocks nor log segments found. Creating new log.
I20260807 11:27:22.180452 2253 tablet_bootstrap.cc:492] T 197a562dcf18493a94ced106edf9e8c5 P e70ceab7bd3b4980a1a35d42a1d27d98: Bootstrap starting.
I20260807 11:27:22.180605 2251 log.cc:824] T 197a562dcf18493a94ced106edf9e8c5 P f1ea6fee5ffb44509a2c2d08b6400fec: Log is configured to *not* fsync() on all Append() calls
I20260807 11:27:22.182649 2252 tablet_bootstrap.cc:654] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a: Neither blocks nor log segments found. Creating new log.
I20260807 11:27:22.184031 2253 tablet_bootstrap.cc:654] T 197a562dcf18493a94ced106edf9e8c5 P e70ceab7bd3b4980a1a35d42a1d27d98: Neither blocks nor log segments found. Creating new log.
I20260807 11:27:22.183954 2252 log.cc:824] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a: Log is configured to *not* fsync() on all Append() calls
I20260807 11:27:22.185392 2253 log.cc:824] T 197a562dcf18493a94ced106edf9e8c5 P e70ceab7bd3b4980a1a35d42a1d27d98: Log is configured to *not* fsync() on all Append() calls
I20260807 11:27:22.185966 2251 tablet_bootstrap.cc:492] T 197a562dcf18493a94ced106edf9e8c5 P f1ea6fee5ffb44509a2c2d08b6400fec: No bootstrap required, opened a new log
I20260807 11:27:22.186303 2251 ts_tablet_manager.cc:1397] T 197a562dcf18493a94ced106edf9e8c5 P f1ea6fee5ffb44509a2c2d08b6400fec: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.007s
I20260807 11:27:22.187453 2252 tablet_bootstrap.cc:492] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a: No bootstrap required, opened a new log
I20260807 11:27:22.187726 2252 ts_tablet_manager.cc:1397] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260807 11:27:22.188166 2253 tablet_bootstrap.cc:492] T 197a562dcf18493a94ced106edf9e8c5 P e70ceab7bd3b4980a1a35d42a1d27d98: No bootstrap required, opened a new log
I20260807 11:27:22.188428 2253 ts_tablet_manager.cc:1397] T 197a562dcf18493a94ced106edf9e8c5 P e70ceab7bd3b4980a1a35d42a1d27d98: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 11:27:22.196142 2253 raft_consensus.cc:348] T 197a562dcf18493a94ced106edf9e8c5 P e70ceab7bd3b4980a1a35d42a1d27d98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 45199 } } peers { permanent_uuid: "21415c113da747bc9d7db1dfd3aa073a" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 42879 } } peers { permanent_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40291 } }
I20260807 11:27:22.196573 2253 raft_consensus.cc:374] T 197a562dcf18493a94ced106edf9e8c5 P e70ceab7bd3b4980a1a35d42a1d27d98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:27:22.196748 2253 raft_consensus.cc:729] T 197a562dcf18493a94ced106edf9e8c5 P e70ceab7bd3b4980a1a35d42a1d27d98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e70ceab7bd3b4980a1a35d42a1d27d98, State: Initialized, Role: FOLLOWER
I20260807 11:27:22.197412 2253 consensus_queue.cc:260] T 197a562dcf18493a94ced106edf9e8c5 P e70ceab7bd3b4980a1a35d42a1d27d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 45199 } } peers { permanent_uuid: "21415c113da747bc9d7db1dfd3aa073a" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 42879 } } peers { permanent_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40291 } }
I20260807 11:27:22.197304 2251 raft_consensus.cc:348] T 197a562dcf18493a94ced106edf9e8c5 P f1ea6fee5ffb44509a2c2d08b6400fec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 45199 } } peers { permanent_uuid: "21415c113da747bc9d7db1dfd3aa073a" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 42879 } } peers { permanent_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40291 } }
I20260807 11:27:22.197844 2251 raft_consensus.cc:374] T 197a562dcf18493a94ced106edf9e8c5 P f1ea6fee5ffb44509a2c2d08b6400fec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:27:22.198010 2251 raft_consensus.cc:729] T 197a562dcf18493a94ced106edf9e8c5 P f1ea6fee5ffb44509a2c2d08b6400fec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1ea6fee5ffb44509a2c2d08b6400fec, State: Initialized, Role: FOLLOWER
I20260807 11:27:22.198626 2235 heartbeater.cc:503] Master 127.16.197.62:46493 was elected leader, sending a full tablet report...
I20260807 11:27:22.198522 2252 raft_consensus.cc:348] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 45199 } } peers { permanent_uuid: "21415c113da747bc9d7db1dfd3aa073a" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 42879 } } peers { permanent_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40291 } }
I20260807 11:27:22.198766 2251 consensus_queue.cc:260] T 197a562dcf18493a94ced106edf9e8c5 P f1ea6fee5ffb44509a2c2d08b6400fec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 45199 } } peers { permanent_uuid: "21415c113da747bc9d7db1dfd3aa073a" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 42879 } } peers { permanent_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40291 } }
I20260807 11:27:22.198956 2252 raft_consensus.cc:374] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:27:22.199098 2252 raft_consensus.cc:729] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21415c113da747bc9d7db1dfd3aa073a, State: Initialized, Role: FOLLOWER
I20260807 11:27:22.199368 2253 ts_tablet_manager.cc:1428] T 197a562dcf18493a94ced106edf9e8c5 P e70ceab7bd3b4980a1a35d42a1d27d98: Time spent starting tablet: real 0.011s user 0.003s sys 0.008s
I20260807 11:27:22.199957 2252 consensus_queue.cc:260] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 45199 } } peers { permanent_uuid: "21415c113da747bc9d7db1dfd3aa073a" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 42879 } } peers { permanent_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40291 } }
I20260807 11:27:22.202127 2252 ts_tablet_manager.cc:1428] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260807 11:27:22.203709 2098 heartbeater.cc:503] Master 127.16.197.62:46493 was elected leader, sending a full tablet report...
I20260807 11:27:22.204396 2251 ts_tablet_manager.cc:1428] T 197a562dcf18493a94ced106edf9e8c5 P f1ea6fee5ffb44509a2c2d08b6400fec: Time spent starting tablet: real 0.018s user 0.010s sys 0.007s
I20260807 11:27:22.245318 2259 raft_consensus.cc:482] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:27:22.245669 2259 raft_consensus.cc:504] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 45199 } } peers { permanent_uuid: "21415c113da747bc9d7db1dfd3aa073a" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 42879 } } peers { permanent_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40291 } }
I20260807 11:27:22.247574 2259 leader_election.cc:288] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1ea6fee5ffb44509a2c2d08b6400fec (127.16.197.2:45199), e70ceab7bd3b4980a1a35d42a1d27d98 (127.16.197.3:40291)
I20260807 11:27:22.254773 2247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:22.248166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:47717 (local address 127.16.197.2:45199)
0807 11:27:22.248381 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 11:27:22.248388 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:27:22.248997 (+ 609us) server_negotiation.cc:407] Connection header received
0807 11:27:22.249249 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:27:22.249255 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:27:22.249325 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:27:22.249430 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:27:22.249984 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.250709 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.252802 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.253790 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.253997 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:27:22.254076 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 11:27:22.254357 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 11:27:22.254483 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 11:27:22.254911 2261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:22.248139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:45199 (local address 127.16.197.1:47717)
0807 11:27:22.248869 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:22.248885 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 11:27:22.249078 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:27:22.249454 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:27:22.249459 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:27:22.249488 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:27:22.249828 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:27:22.249838 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.250861 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.250873 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:27:22.252575 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:27:22.252590 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.254021 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.254028 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:27:22.254192 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:27:22.254215 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 11:27:22.254320 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 11:27:22.254467 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"mutex_wait_us":23,"thread_start_us":892,"threads_started":1}
I20260807 11:27:22.255257 2250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:22.248151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:44871 (local address 127.16.197.3:40291)
0807 11:27:22.248371 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 11:27:22.248378 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:27:22.249491 (+ 1113us) server_negotiation.cc:407] Connection header received
0807 11:27:22.249696 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:27:22.249703 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:27:22.249769 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:27:22.249868 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:27:22.250433 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.251213 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.253173 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.254244 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.254437 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:27:22.254512 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 11:27:22.254863 (+ 351us) server_negotiation.cc:299] Negotiation successful
0807 11:27:22.255012 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 11:27:22.255384 2260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:22.248139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:40291 (local address 127.16.197.1:44871)
0807 11:27:22.249364 (+ 1225us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:22.249378 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 11:27:22.249535 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:27:22.249931 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:27:22.249935 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:27:22.249960 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:27:22.250261 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:27:22.250269 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.251343 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.251351 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:27:22.253020 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:27:22.253031 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.254476 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.254482 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:27:22.254657 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:27:22.254694 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 11:27:22.254839 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 11:27:22.254997 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1109,"mutex_wait_us":53,"thread_start_us":372,"threads_started":1}
I20260807 11:27:22.256306 2051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197a562dcf18493a94ced106edf9e8c5" candidate_uuid: "21415c113da747bc9d7db1dfd3aa073a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" is_pre_election: true
I20260807 11:27:22.256458 2187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197a562dcf18493a94ced106edf9e8c5" candidate_uuid: "21415c113da747bc9d7db1dfd3aa073a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" is_pre_election: true
I20260807 11:27:22.256850 2051 raft_consensus.cc:2454] T 197a562dcf18493a94ced106edf9e8c5 P f1ea6fee5ffb44509a2c2d08b6400fec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21415c113da747bc9d7db1dfd3aa073a in term 0.
I20260807 11:27:22.256960 2187 raft_consensus.cc:2454] T 197a562dcf18493a94ced106edf9e8c5 P e70ceab7bd3b4980a1a35d42a1d27d98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21415c113da747bc9d7db1dfd3aa073a in term 0.
I20260807 11:27:22.257619 1839 leader_election.cc:302] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21415c113da747bc9d7db1dfd3aa073a, f1ea6fee5ffb44509a2c2d08b6400fec; no voters:
I20260807 11:27:22.258031 2259 raft_consensus.cc:2781] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:27:22.258180 2259 raft_consensus.cc:482] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:27:22.258303 2259 raft_consensus.cc:3037] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:27:22.260637 2259 raft_consensus.cc:504] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 45199 } } peers { permanent_uuid: "21415c113da747bc9d7db1dfd3aa073a" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 42879 } } peers { permanent_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40291 } }
I20260807 11:27:22.261283 2259 leader_election.cc:288] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [CANDIDATE]: Term 1 election: Requested vote from peers f1ea6fee5ffb44509a2c2d08b6400fec (127.16.197.2:45199), e70ceab7bd3b4980a1a35d42a1d27d98 (127.16.197.3:40291)
I20260807 11:27:22.261796 2187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197a562dcf18493a94ced106edf9e8c5" candidate_uuid: "21415c113da747bc9d7db1dfd3aa073a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98"
I20260807 11:27:22.261777 2051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197a562dcf18493a94ced106edf9e8c5" candidate_uuid: "21415c113da747bc9d7db1dfd3aa073a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec"
I20260807 11:27:22.262015 2051 raft_consensus.cc:3037] T 197a562dcf18493a94ced106edf9e8c5 P f1ea6fee5ffb44509a2c2d08b6400fec [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:27:22.262010 2187 raft_consensus.cc:3037] T 197a562dcf18493a94ced106edf9e8c5 P e70ceab7bd3b4980a1a35d42a1d27d98 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:27:22.265381 2187 raft_consensus.cc:2454] T 197a562dcf18493a94ced106edf9e8c5 P e70ceab7bd3b4980a1a35d42a1d27d98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21415c113da747bc9d7db1dfd3aa073a in term 1.
I20260807 11:27:22.265374 2051 raft_consensus.cc:2454] T 197a562dcf18493a94ced106edf9e8c5 P f1ea6fee5ffb44509a2c2d08b6400fec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21415c113da747bc9d7db1dfd3aa073a in term 1.
I20260807 11:27:22.265846 1839 leader_election.cc:302] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 21415c113da747bc9d7db1dfd3aa073a, e70ceab7bd3b4980a1a35d42a1d27d98, f1ea6fee5ffb44509a2c2d08b6400fec; no voters:
I20260807 11:27:22.266185 2259 raft_consensus.cc:2781] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:27:22.266654 2259 raft_consensus.cc:686] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [term 1 LEADER]: Becoming Leader. State: Replica: 21415c113da747bc9d7db1dfd3aa073a, State: Running, Role: LEADER
I20260807 11:27:22.267277 2259 consensus_queue.cc:237] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 45199 } } peers { permanent_uuid: "21415c113da747bc9d7db1dfd3aa073a" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 42879 } } peers { permanent_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40291 } }
I20260807 11:27:22.272208 1762 catalog_manager.cc:5495] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a reported cstate change: term changed from 0 to 1, leader changed from <none> to 21415c113da747bc9d7db1dfd3aa073a (127.16.197.1). New cstate: current_term: 1 leader_uuid: "21415c113da747bc9d7db1dfd3aa073a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 45199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21415c113da747bc9d7db1dfd3aa073a" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 42879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40291 } health_report { overall_health: UNKNOWN } } }
I20260807 11:27:22.290320 2249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:22.284601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35232 (local address 127.16.197.1:42879)
0807 11:27:22.284860 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 11:27:22.284868 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:27:22.284889 (+ 21us) server_negotiation.cc:407] Connection header received
0807 11:27:22.285142 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:27:22.285148 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:27:22.285253 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:27:22.285429 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:27:22.285806 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.286564 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.288067 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.288480 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.288618 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:27:22.289128 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:27:22.289708 (+ 580us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:27:22.289880 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 11:27:22.289939 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 11:27:22.290026 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 11:27:22.298022 2099 tablet.cc:2369] T 197a562dcf18493a94ced106edf9e8c5 P f1ea6fee5ffb44509a2c2d08b6400fec: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 11:27:22.298624 2236 tablet.cc:2369] T 197a562dcf18493a94ced106edf9e8c5 P e70ceab7bd3b4980a1a35d42a1d27d98: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:27:22.299085 2051 raft_consensus.cc:1261] T 197a562dcf18493a94ced106edf9e8c5 P f1ea6fee5ffb44509a2c2d08b6400fec [term 1 FOLLOWER]: Refusing update from remote peer 21415c113da747bc9d7db1dfd3aa073a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:27:22.299116 2187 raft_consensus.cc:1261] T 197a562dcf18493a94ced106edf9e8c5 P e70ceab7bd3b4980a1a35d42a1d27d98 [term 1 FOLLOWER]: Refusing update from remote peer 21415c113da747bc9d7db1dfd3aa073a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:27:22.300118 2262 consensus_queue.cc:1035] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:27:22.300479 2259 consensus_queue.cc:1035] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 45199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:27:22.312579 2264 mvcc.cc:205] Tried to move back new op lower bound from 7315873965245464576 to 7315873965130817536. Current Snapshot: MvccSnapshot[applied={T|T < 7315873965245464576}]
I20260807 11:27:22.328481 1762 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:27:22.328765 1762 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:27:22.332556 1762 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272: Sending DeleteTablet for 3 replicas of tablet 197a562dcf18493a94ced106edf9e8c5
I20260807 11:27:22.333774 2029 tablet_service.cc:1514] Processing DeleteTablet for tablet 197a562dcf18493a94ced106edf9e8c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:27:22 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:41716
I20260807 11:27:22.333817 1888 tablet_service.cc:1514] Processing DeleteTablet for tablet 197a562dcf18493a94ced106edf9e8c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:27:22 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:35224
I20260807 11:27:22.334579 2165 tablet_service.cc:1514] Processing DeleteTablet for tablet 197a562dcf18493a94ced106edf9e8c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:27:22 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:45082
I20260807 11:27:22.335052 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 1802
I20260807 11:27:22.357076 2276 tablet_replica.cc:330] T 197a562dcf18493a94ced106edf9e8c5 P e70ceab7bd3b4980a1a35d42a1d27d98: stopping tablet replica
I20260807 11:27:22.357762 2276 raft_consensus.cc:2229] T 197a562dcf18493a94ced106edf9e8c5 P e70ceab7bd3b4980a1a35d42a1d27d98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:27:22.358474 2276 raft_consensus.cc:2258] T 197a562dcf18493a94ced106edf9e8c5 P e70ceab7bd3b4980a1a35d42a1d27d98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:27:22.359097 2249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:22.336653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35234 (local address 127.16.197.1:42879)
0807 11:27:22.336820 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 11:27:22.336825 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:27:22.336844 (+ 19us) server_negotiation.cc:407] Connection header received
0807 11:27:22.336910 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:27:22.336917 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:27:22.337029 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:27:22.337137 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:27:22.339530 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.341047 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.343548 (+ 2501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.344475 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.344615 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:27:22.349085 (+ 4470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:27:22.349104 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:27:22.349128 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:27:22.349182 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:27:22.354332 (+ 5150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:27:22.355473 (+ 1141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:27:22.355491 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:27:22.355497 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:27:22.355708 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:27:22.356266 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:27:22.356274 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:27:22.356280 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:27:22.356796 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:27:22.357112 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0807 11:27:22.358381 (+ 1269us) server_negotiation.cc:299] Negotiation successful
0807 11:27:22.358675 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 11:27:22.360183 1953 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:27:22.361334 2276 ts_tablet_manager.cc:1905] T 197a562dcf18493a94ced106edf9e8c5 P e70ceab7bd3b4980a1a35d42a1d27d98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:27:22.362004 2275 tablet_replica.cc:330] T 197a562dcf18493a94ced106edf9e8c5 P f1ea6fee5ffb44509a2c2d08b6400fec: stopping tablet replica
I20260807 11:27:22.362584 2275 raft_consensus.cc:2229] T 197a562dcf18493a94ced106edf9e8c5 P f1ea6fee5ffb44509a2c2d08b6400fec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:27:22.363268 2275 raft_consensus.cc:2258] T 197a562dcf18493a94ced106edf9e8c5 P f1ea6fee5ffb44509a2c2d08b6400fec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:27:22.365607 2275 ts_tablet_manager.cc:1905] T 197a562dcf18493a94ced106edf9e8c5 P f1ea6fee5ffb44509a2c2d08b6400fec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:27:22.369709 2276 ts_tablet_manager.cc:1918] T 197a562dcf18493a94ced106edf9e8c5 P e70ceab7bd3b4980a1a35d42a1d27d98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:27:22.369925 2276 log.cc:1196] T 197a562dcf18493a94ced106edf9e8c5 P e70ceab7bd3b4980a1a35d42a1d27d98: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/wal/wals/197a562dcf18493a94ced106edf9e8c5
I20260807 11:27:22.370427 2276 ts_tablet_manager.cc:1939] T 197a562dcf18493a94ced106edf9e8c5 P e70ceab7bd3b4980a1a35d42a1d27d98: Deleting consensus metadata
I20260807 11:27:22.372107 1749 catalog_manager.cc:4841] TS e70ceab7bd3b4980a1a35d42a1d27d98 (127.16.197.3:40291): tablet 197a562dcf18493a94ced106edf9e8c5 (table test-table [id=31ab465f3c3c4000bdd2fe8faf6e4afc]) successfully deleted
I20260807 11:27:22.372948 2275 ts_tablet_manager.cc:1918] T 197a562dcf18493a94ced106edf9e8c5 P f1ea6fee5ffb44509a2c2d08b6400fec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:27:22.373133 2275 log.cc:1196] T 197a562dcf18493a94ced106edf9e8c5 P f1ea6fee5ffb44509a2c2d08b6400fec: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/wal/wals/197a562dcf18493a94ced106edf9e8c5
I20260807 11:27:22.373705 2275 ts_tablet_manager.cc:1939] T 197a562dcf18493a94ced106edf9e8c5 P f1ea6fee5ffb44509a2c2d08b6400fec: Deleting consensus metadata
I20260807 11:27:22.375626 1748 catalog_manager.cc:4841] TS f1ea6fee5ffb44509a2c2d08b6400fec (127.16.197.2:45199): tablet 197a562dcf18493a94ced106edf9e8c5 (table test-table [id=31ab465f3c3c4000bdd2fe8faf6e4afc]) successfully deleted
I20260807 11:27:22.462468 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 1802
I20260807 11:27:22.483065 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 1964
W20260807 11:27:22.483146 1751 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.197.1:42879 (error 108)
I20260807 11:27:22.485400 2245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:22.484851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:42879 (local address 127.0.0.1:35236)
0807 11:27:22.485064 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:22.485202 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260807 11:27:22.485795 1751 catalog_manager.cc:4584] TS 21415c113da747bc9d7db1dfd3aa073a (127.16.197.1:42879): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 197a562dcf18493a94ced106edf9e8c5: Network error: Client connection negotiation failed: client connection to 127.16.197.1:42879: connect: Connection refused (error 111)
I20260807 11:27:22.501538 2247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:22.483976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41724 (local address 127.16.197.2:45199)
0807 11:27:22.484181 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 11:27:22.484187 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:27:22.484349 (+ 162us) server_negotiation.cc:407] Connection header received
0807 11:27:22.484563 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:27:22.484569 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:27:22.484657 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:27:22.484763 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:27:22.486487 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.487291 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.488587 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.488960 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.489051 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:27:22.495946 (+ 6895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:27:22.495962 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:27:22.495977 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:27:22.496021 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:27:22.498845 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:27:22.499751 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:27:22.499758 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:27:22.499760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:27:22.499898 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:27:22.500233 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:27:22.500236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:27:22.500238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:27:22.500621 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:27:22.500845 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 11:27:22.501102 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 11:27:22.501278 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 11:27:22.502280 2093 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:27:22.539785 2245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:22.539266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:42879 (local address 127.0.0.1:35248)
0807 11:27:22.539484 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:22.539597 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 11:27:22.540963 2245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:22.540527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:42879 (local address 127.0.0.1:35256)
0807 11:27:22.540694 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:22.540805 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 11:27:22.593537 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 1964
I20260807 11:27:22.601790 2245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:22.601139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:42879 (local address 127.0.0.1:35264)
0807 11:27:22.601420 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:22.601554 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260807 11:27:22.603197 2245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:22.602694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:42879 (local address 127.0.0.1:35278)
0807 11:27:22.602883 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:22.603017 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 11:27:22.613606 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 2101
I20260807 11:27:22.627153 2250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:22.614373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45084 (local address 127.16.197.3:40291)
0807 11:27:22.614533 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 11:27:22.614537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:27:22.614614 (+ 77us) server_negotiation.cc:407] Connection header received
0807 11:27:22.614798 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:27:22.614802 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:27:22.614868 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:27:22.615000 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:27:22.616353 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.617096 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.618476 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:22.618847 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:22.618934 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:27:22.621639 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:27:22.621654 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:27:22.621670 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:27:22.621705 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:27:22.624514 (+ 2809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:27:22.625208 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:27:22.625215 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:27:22.625217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:27:22.625343 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:27:22.625702 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:27:22.625705 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:27:22.625707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:27:22.626047 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:27:22.626252 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 11:27:22.626572 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 11:27:22.626741 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 11:27:22.627818 2229 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:27:22.683619 2245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:22.683127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:42879 (local address 127.0.0.1:35290)
0807 11:27:22.683359 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:22.683458 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 11:27:22.684849 2245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:22.684444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:42879 (local address 127.0.0.1:35302)
0807 11:27:22.684644 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:22.684741 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:42879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260807 11:27:22.719174 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 2101
I20260807 11:27:22.737737 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 1731
I20260807 11:27:22.739073 1795 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:27:22.857940 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 1731
I20260807 11:27:42.883109 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:46493
--webserver_interface=127.16.197.62
--webserver_port=43227
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:45231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:46493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:27:43.235028 2282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:27:43.235378 2282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:27:43.235548 2282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:27:43.242766 2282 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 11:27:43.242913 2282 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 11:27:43.243041 2282 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 11:27:43.243146 2282 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:27:43.243228 2282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:27:43.243355 2282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:27:43.243446 2282 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:27:43.243537 2282 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:27:43.253445 2282 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:45231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:46493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:46493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=43227
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:27:43.254828 2282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:27:43.256230 2282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:27:43.268815 2290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:27:43.268841 2287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:27:43.268868 2288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:27:43.270069 2282 server_base.cc:1034] running on GCE node
I20260807 11:27:43.270848 2282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:27:43.272315 2282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:27:43.273571 2282 hybrid_clock.cc:648] HybridClock initialized: now 1786102063273509 us; error 37 us; skew 500 ppm
I20260807 11:27:43.273916 2282 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:27:43 dist-test-slave-2w3w krb5kdc[1158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102063, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:27:43.277202 2282 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:27:43.278752 2282 webserver.cc:460] Webserver started at http://127.16.197.62:43227/ using document root <none> and password file <none>
I20260807 11:27:43.279536 2282 fs_manager.cc:359] Metadata directory not provided
I20260807 11:27:43.279676 2282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:27:43.285251 2282 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260807 11:27:43.288093 2297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:27:43.289515 2282 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 11:27:43.289726 2282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "818d4fd98b34472384e0e9d8bdf31272"
format_stamp: "Formatted at 2026-08-07 11:26:57 on dist-test-slave-2w3w"
server_key: "018456d574668f87b119ebe7c05d051e"
server_key_iv: "a27613eb1c58dac2ebe84e6d2e3ae5e9"
server_key_version: "encryptionkey@0"
I20260807 11:27:43.290721 2282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:27:43.302618 2282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:27:43.306823 2282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:27:43.307159 2282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:27:43.326428 2282 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:46493
I20260807 11:27:43.326467 2352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:46493 every 8 connection(s)
I20260807 11:27:43.328068 2282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:27:43.332051 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 2282
I20260807 11:27:43.333143 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:42879
--local_ip_for_outbound_sockets=127.16.197.1
--tserver_master_addrs=127.16.197.62:46493
--webserver_port=42255
--webserver_interface=127.16.197.1
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:45231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 11:27:43.336486 2353 sys_catalog.cc:263] Verifying existing consensus state
I20260807 11:27:43.341809 2353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272: Bootstrap starting.
I20260807 11:27:43.363045 2353 log.cc:824] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272: Log is configured to *not* fsync() on all Append() calls
I20260807 11:27:43.384449 2353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 11:27:43.385076 2353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272: Bootstrap complete.
I20260807 11:27:43.394477 2353 raft_consensus.cc:348] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818d4fd98b34472384e0e9d8bdf31272" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46493 } }
I20260807 11:27:43.395678 2353 raft_consensus.cc:729] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 818d4fd98b34472384e0e9d8bdf31272, State: Initialized, Role: FOLLOWER
I20260807 11:27:43.396376 2353 consensus_queue.cc:260] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818d4fd98b34472384e0e9d8bdf31272" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46493 } }
I20260807 11:27:43.396646 2353 raft_consensus.cc:388] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:27:43.396790 2353 raft_consensus.cc:482] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:27:43.396983 2353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [term 2 FOLLOWER]: Advancing to term 3
I20260807 11:27:43.400998 2353 raft_consensus.cc:504] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818d4fd98b34472384e0e9d8bdf31272" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46493 } }
I20260807 11:27:43.401535 2353 leader_election.cc:302] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 818d4fd98b34472384e0e9d8bdf31272; no voters:
I20260807 11:27:43.402284 2353 leader_election.cc:288] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 11:27:43.402438 2357 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 11:27:43.403677 2357 raft_consensus.cc:686] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [term 3 LEADER]: Becoming Leader. State: Replica: 818d4fd98b34472384e0e9d8bdf31272, State: Running, Role: LEADER
I20260807 11:27:43.404309 2357 consensus_queue.cc:237] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818d4fd98b34472384e0e9d8bdf31272" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46493 } }
I20260807 11:27:43.404701 2353 sys_catalog.cc:564] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:27:43.407169 2359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 818d4fd98b34472384e0e9d8bdf31272. Latest consensus state: current_term: 3 leader_uuid: "818d4fd98b34472384e0e9d8bdf31272" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818d4fd98b34472384e0e9d8bdf31272" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46493 } } }
I20260807 11:27:43.407246 2358 sys_catalog.cc:455] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "818d4fd98b34472384e0e9d8bdf31272" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "818d4fd98b34472384e0e9d8bdf31272" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46493 } } }
I20260807 11:27:43.407589 2359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [sys.catalog]: This master's current role is: LEADER
I20260807 11:27:43.407670 2358 sys_catalog.cc:458] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272 [sys.catalog]: This master's current role is: LEADER
I20260807 11:27:43.408499 2363 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:27:43.431689 2363 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:27:43.434082 2363 catalog_manager.cc:1259] Loaded cluster ID: 178fafeb18fe442fbb87a63231c6d5dd
I20260807 11:27:43.434348 2363 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:27:43.439882 2363 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:27:43.442157 2363 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272: Loaded TSK: 0
I20260807 11:27:43.442457 2363 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 818d4fd98b34472384e0e9d8bdf31272: Loaded TSK: 1
F20260807 11:27:43.444238 2363 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f86959779ad google::LogMessage::Fail() at ??:0
@ 0x7f869597a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f86959774de google::LogMessage::Flush() at ??:0
@ 0x7f8695978a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f86996db7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f86996ca73e kudu::security::TokenSigner::ImportKeys() at ??:0
I20260807 11:27:43.480166 2377 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f86ab3a61c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f86ab3a5783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f86ab4312a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f86ab4310c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f86ab47b5e9 std::function<>::operator()() at ??:0
@ 0x7f86ab3a9cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f86ab3a7c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f86ab42f610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f86ab42f439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f86985c3345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f86985c691c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f86985c6799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f86985a1eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f869b87b6db start_thread at ??:0
@ 0x7f869481161f clone at ??:0
W20260807 11:27:43.783797 2355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:27:43.784121 2355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:27:43.784276 2355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:27:43.792013 2355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:27:43.792140 2355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:27:43.792337 2355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:27:43.802582 2355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:45231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:42879
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=42255
--tserver_master_addrs=127.16.197.62:46493
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:27:43.804041 2355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:27:43.805424 2355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:27:43.819149 2401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:27:43.819581 2404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:27:43.819962 2402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:27:43.820298 2355 server_base.cc:1034] running on GCE node
I20260807 11:27:43.820960 2355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:27:43.822003 2355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:27:43.823236 2355 hybrid_clock.cc:648] HybridClock initialized: now 1786102063823181 us; error 48 us; skew 500 ppm
I20260807 11:27:43.823606 2355 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:27:43 dist-test-slave-2w3w krb5kdc[1158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102063, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:27:43.826763 2355 init.cc:377] Logged in from keytab as kudu/127.16.197.1@KRBTEST.COM (short username kudu)
I20260807 11:27:43.828347 2355 webserver.cc:460] Webserver started at http://127.16.197.1:42255/ using document root <none> and password file <none>
I20260807 11:27:43.829083 2355 fs_manager.cc:359] Metadata directory not provided
I20260807 11:27:43.829239 2355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:27:43.835165 2355 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260807 11:27:43.838131 2411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:27:43.839610 2355 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:27:43.839798 2355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "21415c113da747bc9d7db1dfd3aa073a"
format_stamp: "Formatted at 2026-08-07 11:26:57 on dist-test-slave-2w3w"
server_key: "4d911d38a98a222a513f137bce563f03"
server_key_iv: "a5f6e50f6f9f8e830ae985a847475f43"
server_key_version: "encryptionkey@0"
I20260807 11:27:43.840835 2355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:27:43.864526 2355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:27:43.869150 2355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:27:43.869462 2355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:27:43.870512 2355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:27:43.872442 2418 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260807 11:27:43.878393 2355 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 11:27:43.878561 2355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.001s
I20260807 11:27:43.878700 2355 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 11:27:43.882107 2355 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 11:27:43.882247 2355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.000s sys 0.000s
I20260807 11:27:43.882673 2418 tablet_bootstrap.cc:492] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a: Bootstrap starting.
I20260807 11:27:43.917689 2418 log.cc:824] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a: Log is configured to *not* fsync() on all Append() calls
I20260807 11:27:43.924682 2355 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:42879
I20260807 11:27:43.924755 2530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:42879 every 8 connection(s)
I20260807 11:27:43.926321 2355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:27:43.931782 2533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:43.929529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.1:39035)
0807 11:27:43.930505 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:43.930781 (+ 276us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":710,"thread_start_us":473,"threads_started":1}
W20260807 11:27:43.932466 2531 heartbeater.cc:644] Failed to heartbeat to 127.16.197.62:46493 (0 consecutive failures): Network error: Failed to ping master at 127.16.197.62:46493: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
I20260807 11:27:43.933403 2533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:43.933059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.1:38397)
0807 11:27:43.933218 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:43.933296 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 11:27:43.933643 2418 tablet_bootstrap.cc:492] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 11:27:43.934269 2418 tablet_bootstrap.cc:492] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a: Bootstrap complete.
I20260807 11:27:43.934351 2533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:43.933998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.1:40745)
0807 11:27:43.934134 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:43.934219 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 11:27:43.934649 2531 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 11:27:43.935781 2418 ts_tablet_manager.cc:1397] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a: Time spent bootstrapping tablet: real 0.054s user 0.031s sys 0.020s
I20260807 11:27:43.935832 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 2355
I20260807 11:27:43.936487 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:45199
--local_ip_for_outbound_sockets=127.16.197.2
--tserver_master_addrs=127.16.197.62:46493
--webserver_port=35693
--webserver_interface=127.16.197.2
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:45231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 11:27:43.942400 2418 raft_consensus.cc:348] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 45199 } } peers { permanent_uuid: "21415c113da747bc9d7db1dfd3aa073a" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 42879 } } peers { permanent_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40291 } }
I20260807 11:27:43.943463 2418 raft_consensus.cc:729] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21415c113da747bc9d7db1dfd3aa073a, State: Initialized, Role: FOLLOWER
I20260807 11:27:43.944245 2418 consensus_queue.cc:260] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 45199 } } peers { permanent_uuid: "21415c113da747bc9d7db1dfd3aa073a" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 42879 } } peers { permanent_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40291 } }
I20260807 11:27:43.946246 2418 ts_tablet_manager.cc:1428] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a: Time spent starting tablet: real 0.010s user 0.005s sys 0.005s
I20260807 11:27:43.946522 2533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:43.945968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.1:48289)
0807 11:27:43.946194 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:43.946318 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 11:27:43.947641 2533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:43.947177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.1:51501)
0807 11:27:43.947357 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:43.947471 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 11:27:44.288693 2535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:27:44.288995 2535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:27:44.289091 2535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:27:44.296564 2535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:27:44.296687 2535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:27:44.296810 2535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:27:44.306739 2535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:45231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:45199
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=35693
--tserver_master_addrs=127.16.197.62:46493
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:27:44.308095 2535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:27:44.309415 2535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:27:44.323047 2540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:27:44.323021 2543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:27:44.323165 2541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:27:44.324222 2535 server_base.cc:1034] running on GCE node
I20260807 11:27:44.324869 2535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:27:44.325915 2535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:27:44.327128 2535 hybrid_clock.cc:648] HybridClock initialized: now 1786102064327061 us; error 65 us; skew 500 ppm
I20260807 11:27:44.327405 2535 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:27:44 dist-test-slave-2w3w krb5kdc[1158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102064, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:27:44.330651 2535 init.cc:377] Logged in from keytab as kudu/127.16.197.2@KRBTEST.COM (short username kudu)
I20260807 11:27:44.332209 2535 webserver.cc:460] Webserver started at http://127.16.197.2:35693/ using document root <none> and password file <none>
I20260807 11:27:44.332962 2535 fs_manager.cc:359] Metadata directory not provided
I20260807 11:27:44.333117 2535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:27:44.338763 2535 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.000s
I20260807 11:27:44.341888 2552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:27:44.343375 2535 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 11:27:44.343559 2535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "f1ea6fee5ffb44509a2c2d08b6400fec"
format_stamp: "Formatted at 2026-08-07 11:26:58 on dist-test-slave-2w3w"
server_key: "24230e3c61a88550440d9af17bd6e014"
server_key_iv: "3606c81b90291b46c35e81c2a7de2cde"
server_key_version: "encryptionkey@0"
I20260807 11:27:44.344585 2535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:27:44.377315 2535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:27:44.382099 2535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:27:44.382395 2535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:27:44.383512 2535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:27:44.384876 2535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:27:44.384977 2535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:27:44.385138 2535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:27:44.385237 2535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:27:44.419994 2535 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:45199
I20260807 11:27:44.420102 2667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:45199 every 8 connection(s)
I20260807 11:27:44.421624 2535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:27:44.422099 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 2535
I20260807 11:27:44.422986 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:40291
--local_ip_for_outbound_sockets=127.16.197.3
--tserver_master_addrs=127.16.197.62:46493
--webserver_port=38127
--webserver_interface=127.16.197.3
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:45231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 11:27:44.427456 2670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:44.425338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.2:43345)
0807 11:27:44.426217 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:44.426479 (+ 262us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":356,"threads_started":1}
W20260807 11:27:44.428328 2668 heartbeater.cc:644] Failed to heartbeat to 127.16.197.62:46493 (0 consecutive failures): Network error: Failed to ping master at 127.16.197.62:46493: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
I20260807 11:27:44.429343 2670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:44.428967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.2:45541)
0807 11:27:44.429153 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:44.429233 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 11:27:44.430342 2670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:44.430011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.2:53393)
0807 11:27:44.430147 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:44.430222 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 11:27:44.430624 2668 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 11:27:44.766426 2671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:27:44.766701 2671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:27:44.766798 2671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:27:44.774046 2671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:27:44.774154 2671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:27:44.774266 2671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:27:44.784199 2671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:45231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:40291
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=38127
--tserver_master_addrs=127.16.197.62:46493
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:27:44.785434 2671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:27:44.786700 2671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:27:44.800678 2677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:27:44.800765 2679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:27:44.800735 2676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:27:44.801527 2671 server_base.cc:1034] running on GCE node
I20260807 11:27:44.802277 2671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:27:44.803339 2671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:27:44.804579 2671 hybrid_clock.cc:648] HybridClock initialized: now 1786102064804525 us; error 71 us; skew 500 ppm
I20260807 11:27:44.804924 2671 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:27:44 dist-test-slave-2w3w krb5kdc[1158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102064, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:27:44.808135 2671 init.cc:377] Logged in from keytab as kudu/127.16.197.3@KRBTEST.COM (short username kudu)
I20260807 11:27:44.809633 2671 webserver.cc:460] Webserver started at http://127.16.197.3:38127/ using document root <none> and password file <none>
I20260807 11:27:44.810365 2671 fs_manager.cc:359] Metadata directory not provided
I20260807 11:27:44.810530 2671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:27:44.816308 2671 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 11:27:44.819152 2686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:27:44.820503 2671 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:27:44.820691 2671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "e70ceab7bd3b4980a1a35d42a1d27d98"
format_stamp: "Formatted at 2026-08-07 11:26:58 on dist-test-slave-2w3w"
server_key: "3b15e43d3b9a3af0729c9678a90ce33c"
server_key_iv: "e15f28ab910b42670848d0fe34bdb103"
server_key_version: "encryptionkey@0"
I20260807 11:27:44.821717 2671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:27:44.835780 2671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:27:44.839803 2671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:27:44.840083 2671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:27:44.841035 2671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:27:44.842329 2671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:27:44.842450 2671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:27:44.842583 2671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:27:44.842664 2671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:27:44.877012 2671 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:40291
I20260807 11:27:44.877096 2805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:40291 every 8 connection(s)
I20260807 11:27:44.878506 2671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:27:44.883379 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:44.881479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.3:53581)
0807 11:27:44.882193 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:44.882437 (+ 244us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":283,"threads_started":1}
I20260807 11:27:44.883741 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 2671
W20260807 11:27:44.884133 2808 heartbeater.cc:644] Failed to heartbeat to 127.16.197.62:46493 (0 consecutive failures): Network error: Failed to ping master at 127.16.197.62:46493: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
I20260807 11:27:44.885113 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:44.884664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.3:60927)
0807 11:27:44.884843 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:44.884950 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 11:27:44.886088 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:44.885718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.3:52479)
0807 11:27:44.885882 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:44.885975 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 11:27:44.886332 2808 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 11:27:44.949384 2812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:44.948531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.1:38013)
0807 11:27:44.949083 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:44.949175 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":274,"threads_started":1}
I20260807 11:27:45.278560 2813 raft_consensus.cc:482] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:27:45.278895 2813 raft_consensus.cc:504] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 45199 } } peers { permanent_uuid: "21415c113da747bc9d7db1dfd3aa073a" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 42879 } } peers { permanent_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40291 } }
I20260807 11:27:45.280556 2813 leader_election.cc:288] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f1ea6fee5ffb44509a2c2d08b6400fec (127.16.197.2:45199), e70ceab7bd3b4980a1a35d42a1d27d98 (127.16.197.3:40291)
Aug 07 11:27:45 dist-test-slave-2w3w krb5kdc[1158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102063, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for kudu/127.16.197.2@KRBTEST.COM
Aug 07 11:27:45 dist-test-slave-2w3w krb5kdc[1158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102063, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for kudu/127.16.197.3@KRBTEST.COM
I20260807 11:27:45.303300 2816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:45.281375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:49521 (local address 127.16.197.3:40291)
0807 11:27:45.283071 (+ 1696us) server_negotiation.cc:206] Beginning negotiation
0807 11:27:45.283091 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 11:27:45.283380 (+ 289us) server_negotiation.cc:407] Connection header received
0807 11:27:45.284993 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:27:45.285001 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:27:45.285363 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:27:45.285705 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:27:45.287738 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:45.288904 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:45.290882 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:45.291588 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:45.291790 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:27:45.296055 (+ 4265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:27:45.296063 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:27:45.296074 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:27:45.296110 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:27:45.299153 (+ 3043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:27:45.300034 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:27:45.300041 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:27:45.300046 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:27:45.300205 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:27:45.300779 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:27:45.300782 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:27:45.300784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:27:45.301500 (+ 716us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:27:45.301698 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 11:27:45.302490 (+ 792us) server_negotiation.cc:299] Negotiation successful
0807 11:27:45.302852 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1428,"thread_start_us":1189,"threads_started":1}
I20260807 11:27:45.303308 2812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:45.280868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:45199 (local address 127.16.197.1:46429)
0807 11:27:45.281245 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:45.281286 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 11:27:45.282017 (+ 731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:27:45.285864 (+ 3847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:27:45.285877 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:27:45.286747 (+ 870us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:27:45.287537 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:27:45.287561 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:27:45.289116 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:27:45.289120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:27:45.290743 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:27:45.290757 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:27:45.291869 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:27:45.291878 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:27:45.292083 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:27:45.292940 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:27:45.292964 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:27:45.295528 (+ 2564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:27:45.299261 (+ 3733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:27:45.299270 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:27:45.299285 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:27:45.299866 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:27:45.300339 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:27:45.300344 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:27:45.300347 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:27:45.300610 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:27:45.301811 (+ 1201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:27:45.301816 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:27:45.301995 (+ 179us) client_negotiation.cc:769] Sending connection context
0807 11:27:45.302228 (+ 233us) client_negotiation.cc:240] Negotiation successful
0807 11:27:45.302796 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":144,"spinlock_wait_cycles":2176}
I20260807 11:27:45.303357 2814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:45.281117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:46429 (local address 127.16.197.2:45199)
0807 11:27:45.283545 (+ 2428us) server_negotiation.cc:206] Beginning negotiation
0807 11:27:45.283554 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:27:45.283586 (+ 32us) server_negotiation.cc:407] Connection header received
0807 11:27:45.285003 (+ 1417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:27:45.285011 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:27:45.285366 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:27:45.285746 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:27:45.287751 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:45.288917 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:45.290890 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:45.291612 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:45.291816 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:27:45.295849 (+ 4033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:27:45.295857 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:27:45.295872 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:27:45.295915 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:27:45.299108 (+ 3193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:27:45.300028 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:27:45.300042 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:27:45.300047 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:27:45.300209 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:27:45.301101 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:27:45.301105 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:27:45.301107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:27:45.301666 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:27:45.301861 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 11:27:45.302486 (+ 625us) server_negotiation.cc:299] Negotiation successful
0807 11:27:45.302901 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2269,"thread_start_us":2148,"threads_started":1}
I20260807 11:27:45.303478 2815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:45.281100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:40291 (local address 127.16.197.1:49521)
0807 11:27:45.283251 (+ 2151us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:45.283266 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 11:27:45.283415 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:27:45.285884 (+ 2469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:27:45.285890 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:27:45.286747 (+ 857us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:27:45.287536 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:27:45.287561 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:27:45.289120 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:27:45.289125 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:27:45.290743 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:27:45.290756 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:27:45.291817 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:27:45.291827 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:27:45.292080 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:27:45.292870 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:27:45.292897 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:27:45.295786 (+ 2889us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:27:45.299281 (+ 3495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:27:45.299288 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:27:45.299291 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:27:45.299882 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:27:45.300323 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:27:45.300328 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:27:45.300333 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:27:45.300638 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:27:45.301637 (+ 999us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:27:45.301646 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:27:45.301961 (+ 315us) client_negotiation.cc:769] Sending connection context
0807 11:27:45.302228 (+ 267us) client_negotiation.cc:240] Negotiation successful
0807 11:27:45.302986 (+ 758us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2019,"mutex_wait_us":493,"spinlock_wait_cycles":20608,"thread_start_us":1660,"threads_started":1}
W20260807 11:27:45.306699 2413 leader_election.cc:341] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e70ceab7bd3b4980a1a35d42a1d27d98 (127.16.197.3:40291): Not found: Tablet not found: 197a562dcf18493a94ced106edf9e8c5
W20260807 11:27:45.306949 2412 leader_election.cc:341] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f1ea6fee5ffb44509a2c2d08b6400fec (127.16.197.2:45199): Not found: Tablet not found: 197a562dcf18493a94ced106edf9e8c5
I20260807 11:27:45.307157 2412 leader_election.cc:302] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 21415c113da747bc9d7db1dfd3aa073a; no voters: e70ceab7bd3b4980a1a35d42a1d27d98, f1ea6fee5ffb44509a2c2d08b6400fec
I20260807 11:27:45.307590 2813 raft_consensus.cc:2726] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 11:27:45.432094 2817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:45.431216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.2:59725)
0807 11:27:45.431697 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:45.431863 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":223,"threads_started":1}
I20260807 11:27:45.887619 2818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:45.886839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.3:60473)
0807 11:27:45.887301 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:45.887412 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":252,"threads_started":1}
I20260807 11:27:45.951224 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:45.950326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.1:43969)
0807 11:27:45.950787 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:45.950879 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":247,"threads_started":1}
I20260807 11:27:46.433894 2820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:46.433163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.2:47045)
0807 11:27:46.433626 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:46.433734 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":249,"threads_started":1}
I20260807 11:27:46.889683 2821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:46.888805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.3:51031)
0807 11:27:46.889405 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:46.889494 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":303,"threads_started":1}
I20260807 11:27:46.915742 2822 raft_consensus.cc:482] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:27:46.915946 2822 raft_consensus.cc:504] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 45199 } } peers { permanent_uuid: "21415c113da747bc9d7db1dfd3aa073a" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 42879 } } peers { permanent_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40291 } }
I20260807 11:27:46.916530 2822 leader_election.cc:288] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f1ea6fee5ffb44509a2c2d08b6400fec (127.16.197.2:45199), e70ceab7bd3b4980a1a35d42a1d27d98 (127.16.197.3:40291)
W20260807 11:27:46.917785 2413 leader_election.cc:341] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e70ceab7bd3b4980a1a35d42a1d27d98 (127.16.197.3:40291): Not found: Tablet not found: 197a562dcf18493a94ced106edf9e8c5
W20260807 11:27:46.918027 2412 leader_election.cc:341] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f1ea6fee5ffb44509a2c2d08b6400fec (127.16.197.2:45199): Not found: Tablet not found: 197a562dcf18493a94ced106edf9e8c5
I20260807 11:27:46.918134 2412 leader_election.cc:302] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 21415c113da747bc9d7db1dfd3aa073a; no voters: e70ceab7bd3b4980a1a35d42a1d27d98, f1ea6fee5ffb44509a2c2d08b6400fec
I20260807 11:27:46.918354 2822 raft_consensus.cc:2726] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 11:27:46.953296 2823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:46.952209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.1:34199)
0807 11:27:46.952886 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:46.953005 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":283,"threads_started":1}
I20260807 11:27:47.435719 2824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:47.434945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.2:54843)
0807 11:27:47.435449 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:47.435536 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":228,"threads_started":1}
I20260807 11:27:47.891373 2825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:47.890532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.3:60967)
0807 11:27:47.891092 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:47.891202 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":265,"threads_started":1}
I20260807 11:27:47.954855 2826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:47.954168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.1:37763)
0807 11:27:47.954578 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:47.954657 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":213,"threads_started":1}
I20260807 11:27:48.437390 2827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:48.436660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.2:40109)
0807 11:27:48.437122 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:48.437208 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":223,"threads_started":1}
I20260807 11:27:48.893291 2828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:48.892492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.3:40023)
0807 11:27:48.892982 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:48.893086 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":228,"threads_started":1}
I20260807 11:27:48.956790 2829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:48.955983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.1:58941)
0807 11:27:48.956472 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:48.956569 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":275,"threads_started":1}
I20260807 11:27:49.058853 2830 raft_consensus.cc:482] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:27:49.059154 2830 raft_consensus.cc:504] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 45199 } } peers { permanent_uuid: "21415c113da747bc9d7db1dfd3aa073a" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 42879 } } peers { permanent_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40291 } }
I20260807 11:27:49.059907 2830 leader_election.cc:288] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f1ea6fee5ffb44509a2c2d08b6400fec (127.16.197.2:45199), e70ceab7bd3b4980a1a35d42a1d27d98 (127.16.197.3:40291)
W20260807 11:27:49.060979 2413 leader_election.cc:341] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e70ceab7bd3b4980a1a35d42a1d27d98 (127.16.197.3:40291): Not found: Tablet not found: 197a562dcf18493a94ced106edf9e8c5
W20260807 11:27:49.061195 2412 leader_election.cc:341] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f1ea6fee5ffb44509a2c2d08b6400fec (127.16.197.2:45199): Not found: Tablet not found: 197a562dcf18493a94ced106edf9e8c5
I20260807 11:27:49.061305 2412 leader_election.cc:302] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 21415c113da747bc9d7db1dfd3aa073a; no voters: e70ceab7bd3b4980a1a35d42a1d27d98, f1ea6fee5ffb44509a2c2d08b6400fec
I20260807 11:27:49.061573 2830 raft_consensus.cc:2726] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 11:27:49.439227 2831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:49.438373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.2:43159)
0807 11:27:49.438890 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:49.439055 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":271,"threads_started":1}
I20260807 11:27:49.895164 2832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:49.894281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.3:38925)
0807 11:27:49.894864 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:49.894999 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":315,"threads_started":1}
I20260807 11:27:49.958465 2833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:49.957678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.1:36645)
0807 11:27:49.958181 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:49.958279 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":288,"threads_started":1}
I20260807 11:27:50.440966 2834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:50.440165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.2:48305)
0807 11:27:50.440701 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:50.440780 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":308,"threads_started":1}
I20260807 11:27:50.860322 2835 raft_consensus.cc:482] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:27:50.860575 2835 raft_consensus.cc:504] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 45199 } } peers { permanent_uuid: "21415c113da747bc9d7db1dfd3aa073a" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 42879 } } peers { permanent_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40291 } }
I20260807 11:27:50.861268 2835 leader_election.cc:288] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f1ea6fee5ffb44509a2c2d08b6400fec (127.16.197.2:45199), e70ceab7bd3b4980a1a35d42a1d27d98 (127.16.197.3:40291)
W20260807 11:27:50.862246 2413 leader_election.cc:341] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e70ceab7bd3b4980a1a35d42a1d27d98 (127.16.197.3:40291): Not found: Tablet not found: 197a562dcf18493a94ced106edf9e8c5
W20260807 11:27:50.862449 2412 leader_election.cc:341] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f1ea6fee5ffb44509a2c2d08b6400fec (127.16.197.2:45199): Not found: Tablet not found: 197a562dcf18493a94ced106edf9e8c5
I20260807 11:27:50.862555 2412 leader_election.cc:302] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 21415c113da747bc9d7db1dfd3aa073a; no voters: e70ceab7bd3b4980a1a35d42a1d27d98, f1ea6fee5ffb44509a2c2d08b6400fec
I20260807 11:27:50.862860 2835 raft_consensus.cc:2726] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 11:27:50.897640 2836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:50.896208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.3:51725)
0807 11:27:50.897265 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:50.897382 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":886,"thread_start_us":715,"threads_started":1}
I20260807 11:27:50.960243 2837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:50.959467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.1:42101)
0807 11:27:50.959944 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:50.960045 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":237,"threads_started":1}
I20260807 11:27:51.442911 2838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:51.441942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.2:38235)
0807 11:27:51.442627 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:51.442721 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":343,"threads_started":1}
I20260807 11:27:51.899718 2839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:51.898793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.3:51931)
0807 11:27:51.899405 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:51.899500 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":282,"threads_started":1}
I20260807 11:27:51.962275 2840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:51.961507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.1:51679)
0807 11:27:51.962008 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:51.962102 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":272,"threads_started":1}
I20260807 11:27:52.444612 2841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:52.443919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.2:52051)
0807 11:27:52.444335 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:52.444417 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":222,"threads_started":1}
I20260807 11:27:52.901443 2842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:52.900686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.3:39927)
0807 11:27:52.901151 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:52.901240 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":246,"threads_started":1}
I20260807 11:27:52.964250 2843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:52.963424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.1:52835)
0807 11:27:52.963955 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:52.964052 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":275,"threads_started":1}
I20260807 11:27:53.446264 2844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:53.445491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.2:46327)
0807 11:27:53.446018 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:53.446106 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":254,"threads_started":1}
I20260807 11:27:53.903363 2845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:53.902511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.3:47455)
0807 11:27:53.903115 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:53.903203 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":315,"threads_started":1}
I20260807 11:27:53.966233 2846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:53.965437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.1:37905)
0807 11:27:53.965969 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:53.966059 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":313,"threads_started":1}
I20260807 11:27:54.448408 2847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:54.447525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.2:50809)
0807 11:27:54.448058 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:54.448224 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":276,"threads_started":1}
I20260807 11:27:54.905274 2848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:54.904464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.3:59309)
0807 11:27:54.904986 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:54.905076 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":281,"threads_started":1}
I20260807 11:27:54.968325 2849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:54.967318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.1:46019)
0807 11:27:54.967963 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:54.968076 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":337,"threads_started":1}
I20260807 11:27:55.450126 2850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:55.449341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.2:56395)
0807 11:27:55.449848 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:55.449960 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":261,"threads_started":1}
I20260807 11:27:55.907197 2851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:55.906291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.3:41247)
0807 11:27:55.906834 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:55.906931 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":261,"threads_started":1}
I20260807 11:27:55.970225 2852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:55.969360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.1:45947)
0807 11:27:55.969948 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:55.970047 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":329,"threads_started":1}
I20260807 11:27:56.451926 2853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:56.451160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.2:42285)
0807 11:27:56.451621 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:56.451709 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":232,"threads_started":1}
I20260807 11:27:56.909049 2854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:56.908188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.3:57393)
0807 11:27:56.908743 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:56.908832 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":304,"threads_started":1}
I20260807 11:27:56.972348 2855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:56.971447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.1:41257)
0807 11:27:56.972031 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:56.972134 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":267,"threads_started":1}
I20260807 11:27:57.216764 2856 raft_consensus.cc:482] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:27:57.217051 2856 raft_consensus.cc:504] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1ea6fee5ffb44509a2c2d08b6400fec" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 45199 } } peers { permanent_uuid: "21415c113da747bc9d7db1dfd3aa073a" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 42879 } } peers { permanent_uuid: "e70ceab7bd3b4980a1a35d42a1d27d98" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40291 } }
I20260807 11:27:57.217810 2856 leader_election.cc:288] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f1ea6fee5ffb44509a2c2d08b6400fec (127.16.197.2:45199), e70ceab7bd3b4980a1a35d42a1d27d98 (127.16.197.3:40291)
W20260807 11:27:57.218904 2412 leader_election.cc:341] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f1ea6fee5ffb44509a2c2d08b6400fec (127.16.197.2:45199): Not found: Tablet not found: 197a562dcf18493a94ced106edf9e8c5
W20260807 11:27:57.219146 2413 leader_election.cc:341] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e70ceab7bd3b4980a1a35d42a1d27d98 (127.16.197.3:40291): Not found: Tablet not found: 197a562dcf18493a94ced106edf9e8c5
I20260807 11:27:57.219291 2413 leader_election.cc:302] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 21415c113da747bc9d7db1dfd3aa073a; no voters: e70ceab7bd3b4980a1a35d42a1d27d98, f1ea6fee5ffb44509a2c2d08b6400fec
I20260807 11:27:57.219551 2856 raft_consensus.cc:2726] T 197a562dcf18493a94ced106edf9e8c5 P 21415c113da747bc9d7db1dfd3aa073a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 11:27:57.454027 2857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:57.453180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.2:60673)
0807 11:27:57.453727 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:57.453846 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":277,"threads_started":1}
I20260807 11:27:57.910766 2858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:57.910010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.3:34421)
0807 11:27:57.910500 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:57.910598 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":243,"threads_started":1}
I20260807 11:27:57.974225 2859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:57.973462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.1:48851)
0807 11:27:57.973916 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:57.974005 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":234,"threads_started":1}
I20260807 11:27:58.455856 2860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:58.455107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.2:37207)
0807 11:27:58.455561 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:58.455649 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":235,"threads_started":1}
I20260807 11:27:58.912793 2861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:58.911922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.3:51929)
0807 11:27:58.912493 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:58.912586 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":293,"threads_started":1}
I20260807 11:27:58.976575 2862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:58.975765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.1:37315)
0807 11:27:58.976304 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:58.976384 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":269,"threads_started":1}
I20260807 11:27:59.457832 2863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:59.457083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.2:43637)
0807 11:27:59.457543 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:59.457637 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":231,"threads_started":1}
I20260807 11:27:59.886477 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 2355
I20260807 11:27:59.906517 2864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:59.887744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45718 (local address 127.16.197.1:42879)
0807 11:27:59.888478 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0807 11:27:59.888485 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:27:59.888512 (+ 27us) server_negotiation.cc:407] Connection header received
0807 11:27:59.888607 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:27:59.888615 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:27:59.888835 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:27:59.889020 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:27:59.890583 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:59.891668 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:59.892919 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:27:59.893499 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:27:59.893642 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:27:59.900378 (+ 6736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:27:59.900392 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:27:59.900407 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:27:59.900444 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:27:59.903466 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:27:59.904232 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:27:59.904240 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:27:59.904242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:27:59.904379 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:27:59.904790 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:27:59.904796 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:27:59.904799 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:27:59.905194 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:27:59.905524 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0807 11:27:59.905868 (+ 344us) server_negotiation.cc:299] Negotiation successful
0807 11:27:59.906185 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":447,"threads_started":1}
I20260807 11:27:59.907908 2526 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:27:59.915071 2866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:27:59.914104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.3:33373)
0807 11:27:59.914640 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0807 11:27:59.914824 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":247,"threads_started":1}
I20260807 11:28:00.003993 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 2355
I20260807 11:28:00.004030 2867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:00.002664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:46493 (local address 127.16.197.1:54525)
0807 11:28:00.003342 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:00.003528 (+ 186us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.62:46493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":530,"threads_started":1}
I20260807 11:28:00.026046 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 2535
I20260807 11:28:00.040412 2868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:00.026804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34250 (local address 127.16.197.2:45199)
0807 11:28:00.027417 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:00.027429 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:00.027444 (+ 15us) server_negotiation.cc:407] Connection header received
0807 11:28:00.027498 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:00.027501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:00.027558 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:00.027683 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:00.029086 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:00.029846 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:00.031408 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:00.031861 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:00.032008 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:00.034624 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:00.034629 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:00.034632 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:00.034668 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:00.037556 (+ 2888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:00.038264 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:00.038271 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:00.038275 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:00.038416 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:00.038837 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:00.038843 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:00.038847 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:00.039188 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:00.039492 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:00.039795 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 11:28:00.040034 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":373,"threads_started":1}
I20260807 11:28:00.041109 2663 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:00.128209 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 2535
I20260807 11:28:00.149106 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 2671
I20260807 11:28:00.163393 2870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:00.149989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45292 (local address 127.16.197.3:40291)
0807 11:28:00.150619 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:00.150624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:00.150643 (+ 19us) server_negotiation.cc:407] Connection header received
0807 11:28:00.150698 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:00.150703 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:00.150775 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:00.150885 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:00.152413 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:00.153274 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:00.154474 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:00.154938 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:00.155237 (+ 299us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:00.157891 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:00.157896 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:00.157899 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:00.157933 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:00.160859 (+ 2926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:00.161519 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:00.161523 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:00.161525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:00.161670 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:00.162002 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:00.162005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:00.162007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:00.162260 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:00.162476 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:00.162700 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 11:28:00.162902 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":352,"threads_started":1}
I20260807 11:28:00.164059 2801 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:00.249310 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 2671
2026-08-07T11:28:00Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64954 ms)
[----------] 30 tests from SecurityITest (314275 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:28:00 dist-test-slave-2w3w krb5kdc[2875](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:28:00 dist-test-slave-2w3w krb5kdc[2875](info): set up 2 sockets
Aug 07 11:28:00 dist-test-slave-2w3w krb5kdc[2875](info): commencing operation
krb5kdc: starting...
W20260807 11:28:01.409338 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.001s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:28:01 dist-test-slave-2w3w krb5kdc[2875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:28:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:28:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:28:01.699120 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:38263
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:33975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:38263
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:02.049706 2889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:02.050055 2889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:02.050212 2889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:02.057480 2889 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:28:02.057631 2889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:02.057734 2889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:02.057838 2889 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:28:02.057924 2889 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:28:02.067932 2889 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:33975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:38263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:38263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:02.069352 2889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:02.070721 2889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:02.083235 2894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:02.083303 2895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:02.083355 2897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:02.084095 2889 server_base.cc:1034] running on GCE node
I20260807 11:28:02.084848 2889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:02.086261 2889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:02.087510 2889 hybrid_clock.cc:648] HybridClock initialized: now 1786102082087467 us; error 32 us; skew 500 ppm
I20260807 11:28:02.087883 2889 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:02 dist-test-slave-2w3w krb5kdc[2875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102082, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:02.091209 2889 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:28:02.092799 2889 webserver.cc:460] Webserver started at http://127.16.197.62:41561/ using document root <none> and password file <none>
I20260807 11:28:02.093519 2889 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:02.093667 2889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:02.093993 2889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:02.097239 2889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "a9a6789029734079944ec94637f552b4"
format_stamp: "Formatted at 2026-08-07 11:28:02 on dist-test-slave-2w3w"
server_key: "db387c28edd039bc45fe8278064657b8"
server_key_iv: "a011fc992a42745aab6d01b3d66e4bce"
server_key_version: "encryptionkey@0"
I20260807 11:28:02.098075 2889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "a9a6789029734079944ec94637f552b4"
format_stamp: "Formatted at 2026-08-07 11:28:02 on dist-test-slave-2w3w"
server_key: "db387c28edd039bc45fe8278064657b8"
server_key_iv: "a011fc992a42745aab6d01b3d66e4bce"
server_key_version: "encryptionkey@0"
I20260807 11:28:02.103642 2889 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 11:28:02.107324 2904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:02.108791 2889 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 11:28:02.108999 2889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "a9a6789029734079944ec94637f552b4"
format_stamp: "Formatted at 2026-08-07 11:28:02 on dist-test-slave-2w3w"
server_key: "db387c28edd039bc45fe8278064657b8"
server_key_iv: "a011fc992a42745aab6d01b3d66e4bce"
server_key_version: "encryptionkey@0"
I20260807 11:28:02.109229 2889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:02.130571 2889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:02.135357 2889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:02.135665 2889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:02.155637 2889 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:38263
I20260807 11:28:02.155676 2958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:38263 every 8 connection(s)
I20260807 11:28:02.157073 2889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:28:02.158010 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 2889
I20260807 11:28:02.158267 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:28:02.158947 17172 external_mini_cluster.cc:1428] Setting key f1125602c7fa13966fd4a8522c6c7d92
I20260807 11:28:02.161566 2959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 11:28:02 dist-test-slave-2w3w krb5kdc[2875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:28:02.175693 2959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9a6789029734079944ec94637f552b4: Bootstrap starting.
I20260807 11:28:02.179327 2959 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9a6789029734079944ec94637f552b4: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:02.180397 2959 log.cc:824] T 00000000000000000000000000000000 P a9a6789029734079944ec94637f552b4: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:02.183955 2959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9a6789029734079944ec94637f552b4: No bootstrap required, opened a new log
I20260807 11:28:02.184284 2963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:02.161214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57436 (local address 127.16.197.62:38263)
0807 11:28:02.162069 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:02.162080 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:02.162122 (+ 42us) server_negotiation.cc:407] Connection header received
0807 11:28:02.163912 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:02.163935 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:02.164326 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:02.164658 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:02.166657 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:02.168113 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:02.170028 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:02.170700 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:02.170859 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:02.176026 (+ 5167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:02.176036 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:02.176052 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:02.176092 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:02.179466 (+ 3374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:02.180281 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:02.180292 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:02.180305 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:02.180491 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:02.181022 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:02.181029 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:02.181035 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:02.181944 (+ 909us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:02.182191 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:02.182699 (+ 508us) server_negotiation.cc:299] Negotiation successful
0807 11:28:02.183221 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":395,"threads_started":1}
I20260807 11:28:02.192323 2959 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9a6789029734079944ec94637f552b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9a6789029734079944ec94637f552b4" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 38263 } }
I20260807 11:28:02.192734 2959 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9a6789029734079944ec94637f552b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:02.192848 2959 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9a6789029734079944ec94637f552b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a6789029734079944ec94637f552b4, State: Initialized, Role: FOLLOWER
I20260807 11:28:02.193414 2959 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9a6789029734079944ec94637f552b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9a6789029734079944ec94637f552b4" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 38263 } }
I20260807 11:28:02.193660 2959 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9a6789029734079944ec94637f552b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:28:02.193806 2959 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9a6789029734079944ec94637f552b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:28:02.194003 2959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9a6789029734079944ec94637f552b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:02.196626 2959 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9a6789029734079944ec94637f552b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9a6789029734079944ec94637f552b4" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 38263 } }
I20260807 11:28:02.197142 2959 leader_election.cc:302] T 00000000000000000000000000000000 P a9a6789029734079944ec94637f552b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9a6789029734079944ec94637f552b4; no voters:
I20260807 11:28:02.197728 2959 leader_election.cc:288] T 00000000000000000000000000000000 P a9a6789029734079944ec94637f552b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:28:02.197955 2965 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9a6789029734079944ec94637f552b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:02.198683 2965 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9a6789029734079944ec94637f552b4 [term 1 LEADER]: Becoming Leader. State: Replica: a9a6789029734079944ec94637f552b4, State: Running, Role: LEADER
I20260807 11:28:02.199360 2965 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9a6789029734079944ec94637f552b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9a6789029734079944ec94637f552b4" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 38263 } }
I20260807 11:28:02.200142 2959 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9a6789029734079944ec94637f552b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:28:02.202724 2967 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9a6789029734079944ec94637f552b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9a6789029734079944ec94637f552b4. Latest consensus state: current_term: 1 leader_uuid: "a9a6789029734079944ec94637f552b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9a6789029734079944ec94637f552b4" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 38263 } } }
I20260807 11:28:02.203208 2967 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9a6789029734079944ec94637f552b4 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:02.203824 2966 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9a6789029734079944ec94637f552b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9a6789029734079944ec94637f552b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9a6789029734079944ec94637f552b4" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 38263 } } }
I20260807 11:28:02.204274 2966 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9a6789029734079944ec94637f552b4 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:02.207509 2975 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:28:02.212939 2975 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:28:02.222461 2975 catalog_manager.cc:1347] Generated new cluster ID: 389043fb05784962838b7fe379f35c39
I20260807 11:28:02.222602 2975 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:28:02.253247 2975 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:28:02.254645 2975 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:28:02.264890 2975 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9a6789029734079944ec94637f552b4: Generated new TSK 0
I20260807 11:28:02.265774 2975 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.197.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
I20260807 11:28:02.338377 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:0
--local_ip_for_outbound_sockets=127.16.197.1
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:38263
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:33975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:02.693634 2987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:02.693960 2987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:02.694149 2987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:02.702046 2987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:02.702211 2987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:02.702410 2987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:28:02.713163 2987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:33975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:38263
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:02.714689 2987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:02.716153 2987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:02.729681 2992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:02.729619 2995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:02.729707 2993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:02.730683 2987 server_base.cc:1034] running on GCE node
I20260807 11:28:02.731442 2987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:02.732460 2987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:02.733740 2987 hybrid_clock.cc:648] HybridClock initialized: now 1786102082733681 us; error 58 us; skew 500 ppm
I20260807 11:28:02.734086 2987 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:02 dist-test-slave-2w3w krb5kdc[2875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102082, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:02.737486 2987 init.cc:377] Logged in from keytab as kudu/127.16.197.1@KRBTEST.COM (short username kudu)
I20260807 11:28:02.739082 2987 webserver.cc:460] Webserver started at http://127.16.197.1:42279/ using document root <none> and password file <none>
I20260807 11:28:02.739805 2987 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:02.739955 2987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:02.740285 2987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:02.743367 2987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-0/data/instance:
uuid: "226ded7d1c91473ebd5aad855f22ea37"
format_stamp: "Formatted at 2026-08-07 11:28:02 on dist-test-slave-2w3w"
server_key: "c1919d26bc2cb67a876290b97254bbe9"
server_key_iv: "6b66086c92537589a641654c7a44c313"
server_key_version: "encryptionkey@0"
I20260807 11:28:02.744165 2987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance:
uuid: "226ded7d1c91473ebd5aad855f22ea37"
format_stamp: "Formatted at 2026-08-07 11:28:02 on dist-test-slave-2w3w"
server_key: "c1919d26bc2cb67a876290b97254bbe9"
server_key_iv: "6b66086c92537589a641654c7a44c313"
server_key_version: "encryptionkey@0"
I20260807 11:28:02.749498 2987 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 11:28:02.753006 3002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:02.754390 2987 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260807 11:28:02.754583 2987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "226ded7d1c91473ebd5aad855f22ea37"
format_stamp: "Formatted at 2026-08-07 11:28:02 on dist-test-slave-2w3w"
server_key: "c1919d26bc2cb67a876290b97254bbe9"
server_key_iv: "6b66086c92537589a641654c7a44c313"
server_key_version: "encryptionkey@0"
I20260807 11:28:02.754817 2987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:02.772596 2987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:02.776808 2987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:02.777113 2987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:02.778121 2987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:28:02.779668 2987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:28:02.779774 2987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:02.779928 2987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:28:02.780002 2987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:02.814385 2987 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:34535
I20260807 11:28:02.814443 3122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:34535 every 8 connection(s)
I20260807 11:28:02.815785 2987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:28:02.816988 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 2987
I20260807 11:28:02.817209 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance
I20260807 11:28:02.817939 17172 external_mini_cluster.cc:1428] Setting key ebbbb70c96069c50ad48ba93587e91c3
Aug 07 11:28:02 dist-test-slave-2w3w krb5kdc[2875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102082, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:28:02.836570 3125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:02.818527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:56159 (local address 127.16.197.62:38263)
0807 11:28:02.819092 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:02.819098 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:02.820303 (+ 1205us) server_negotiation.cc:407] Connection header received
0807 11:28:02.822219 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:02.822225 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:02.822326 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:02.822444 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:02.824098 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:02.824994 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:02.826596 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:02.827068 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:02.827210 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:02.830718 (+ 3508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:02.830723 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:02.830726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:02.830758 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:02.833363 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:02.834130 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:02.834138 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:02.834142 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:02.834277 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:02.834732 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:02.834739 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:02.834742 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:02.835076 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:02.835234 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:02.835953 (+ 719us) server_negotiation.cc:299] Negotiation successful
0807 11:28:02.836183 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":272,"threads_started":1}
I20260807 11:28:02.837549 3126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:02.819268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:38263 (local address 127.16.197.1:56159)
0807 11:28:02.820130 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:02.820180 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:02.821956 (+ 1776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:02.822695 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:02.822704 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:02.823338 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:28:02.823922 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:02.823939 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:02.825152 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:02.825158 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:02.826446 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:02.826458 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:02.827323 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:02.827331 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:02.827565 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:02.828274 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:28:02.828303 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:28:02.830493 (+ 2190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:28:02.833531 (+ 3038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:28:02.833540 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:28:02.833556 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:28:02.834000 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:28:02.834399 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:28:02.834404 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:28:02.834407 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:28:02.834603 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:28:02.835204 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:28:02.835212 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:28:02.835546 (+ 334us) client_negotiation.cc:769] Sending connection context
0807 11:28:02.835871 (+ 325us) client_negotiation.cc:240] Negotiation successful
0807 11:28:02.836408 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":332,"threads_started":1}
I20260807 11:28:02.839229 3123 heartbeater.cc:347] Connected to a master server at 127.16.197.62:38263
I20260807 11:28:02.839574 3123 heartbeater.cc:464] Registering TS with master...
I20260807 11:28:02.840309 3123 heartbeater.cc:511] Master 127.16.197.62:38263 requested a full tablet report, sending...
I20260807 11:28:02.842506 2923 ts_manager.cc:194] Registered new tserver with Master: 226ded7d1c91473ebd5aad855f22ea37 (127.16.197.1:34535)
I20260807 11:28:02.843983 2923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.1@KRBTEST.COM'} at 127.16.197.1:56159
WARNING: no policy specified for kudu/127.16.197.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
I20260807 11:28:02.881798 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:0
--local_ip_for_outbound_sockets=127.16.197.2
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:38263
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:33975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:03.224335 3129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:03.224622 3129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:03.224794 3129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:03.232194 3129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:03.232300 3129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:03.232460 3129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:28:03.242266 3129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:33975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:38263
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:03.243714 3129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:03.245048 3129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:03.259433 3134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:03.259395 3135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:03.259344 3137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:03.260262 3129 server_base.cc:1034] running on GCE node
I20260807 11:28:03.260870 3129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:03.261816 3129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:03.263068 3129 hybrid_clock.cc:648] HybridClock initialized: now 1786102083263017 us; error 42 us; skew 500 ppm
I20260807 11:28:03.263398 3129 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:03 dist-test-slave-2w3w krb5kdc[2875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102083, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:03.266489 3129 init.cc:377] Logged in from keytab as kudu/127.16.197.2@KRBTEST.COM (short username kudu)
I20260807 11:28:03.268046 3129 webserver.cc:460] Webserver started at http://127.16.197.2:37515/ using document root <none> and password file <none>
I20260807 11:28:03.268762 3129 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:03.268910 3129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:03.269240 3129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:03.272188 3129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-1/data/instance:
uuid: "e8fa390f893444618de726274a38864a"
format_stamp: "Formatted at 2026-08-07 11:28:03 on dist-test-slave-2w3w"
server_key: "5d1a28bf75b5b63444b2b9f4fe6f9ad4"
server_key_iv: "b8eb2e8a4ede87d460d1834f1beeb629"
server_key_version: "encryptionkey@0"
I20260807 11:28:03.272964 3129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance:
uuid: "e8fa390f893444618de726274a38864a"
format_stamp: "Formatted at 2026-08-07 11:28:03 on dist-test-slave-2w3w"
server_key: "5d1a28bf75b5b63444b2b9f4fe6f9ad4"
server_key_iv: "b8eb2e8a4ede87d460d1834f1beeb629"
server_key_version: "encryptionkey@0"
I20260807 11:28:03.278154 3129 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260807 11:28:03.281587 3144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:03.282852 3129 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 11:28:03.283075 3129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "e8fa390f893444618de726274a38864a"
format_stamp: "Formatted at 2026-08-07 11:28:03 on dist-test-slave-2w3w"
server_key: "5d1a28bf75b5b63444b2b9f4fe6f9ad4"
server_key_iv: "b8eb2e8a4ede87d460d1834f1beeb629"
server_key_version: "encryptionkey@0"
I20260807 11:28:03.283285 3129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:03.297127 3129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:03.301522 3129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:03.301851 3129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:03.302911 3129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:28:03.304383 3129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:28:03.304487 3129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:03.304648 3129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:28:03.304724 3129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:03.338678 3129 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:38477
I20260807 11:28:03.338766 3262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:38477 every 8 connection(s)
I20260807 11:28:03.340030 3129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:28:03.350319 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 3129
I20260807 11:28:03.350541 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance
I20260807 11:28:03.351167 17172 external_mini_cluster.cc:1428] Setting key 773002955f9f9c1e6e9893ded445b0fe
Aug 07 11:28:03 dist-test-slave-2w3w krb5kdc[2875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102083, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:28:03.360477 3265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:03.342595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:52953 (local address 127.16.197.62:38263)
0807 11:28:03.343091 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:03.343097 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:03.344156 (+ 1059us) server_negotiation.cc:407] Connection header received
0807 11:28:03.345897 (+ 1741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:03.345901 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:03.345986 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:03.346085 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:03.347644 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:03.348393 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:03.349916 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:03.350597 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:03.350736 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:03.354480 (+ 3744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:03.354485 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:03.354489 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:03.354525 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:03.357257 (+ 2732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:03.358036 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:03.358045 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:03.358048 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:03.358186 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:03.358653 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:03.358658 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:03.358668 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:03.358945 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:03.359155 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:03.359861 (+ 706us) server_negotiation.cc:299] Negotiation successful
0807 11:28:03.360102 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":213,"threads_started":1}
I20260807 11:28:03.361492 3266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:03.343256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:38263 (local address 127.16.197.2:52953)
0807 11:28:03.343983 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:03.344026 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:03.345645 (+ 1619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:03.346296 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:03.346304 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:03.346820 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:28:03.347483 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:03.347500 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:03.348531 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:03.348535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:03.349766 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:03.349778 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:03.350796 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:03.350801 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:03.351077 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:03.351811 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:28:03.351834 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:28:03.354246 (+ 2412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:28:03.357437 (+ 3191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:28:03.357447 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:28:03.357466 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:28:03.357904 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:28:03.358312 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:28:03.358316 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:28:03.358320 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:28:03.358543 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:28:03.359123 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:28:03.359131 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:28:03.359495 (+ 364us) client_negotiation.cc:769] Sending connection context
0807 11:28:03.359758 (+ 263us) client_negotiation.cc:240] Negotiation successful
0807 11:28:03.360310 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":287,"threads_started":1}
I20260807 11:28:03.362947 3263 heartbeater.cc:347] Connected to a master server at 127.16.197.62:38263
I20260807 11:28:03.363312 3263 heartbeater.cc:464] Registering TS with master...
I20260807 11:28:03.364207 3263 heartbeater.cc:511] Master 127.16.197.62:38263 requested a full tablet report, sending...
I20260807 11:28:03.365792 2923 ts_manager.cc:194] Registered new tserver with Master: e8fa390f893444618de726274a38864a (127.16.197.2:38477)
I20260807 11:28:03.366616 2923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.2@KRBTEST.COM'} at 127.16.197.2:52953
WARNING: no policy specified for kudu/127.16.197.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
I20260807 11:28:03.414328 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:0
--local_ip_for_outbound_sockets=127.16.197.3
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:38263
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:33975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:03.758271 3269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:03.758625 3269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:03.758819 3269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:03.766381 3269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:03.766512 3269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:03.766692 3269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:28:03.776675 3269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:33975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:38263
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:03.778048 3269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:03.779613 3269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:03.793870 3274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:03.793906 3277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:03.793951 3275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:03.795249 3269 server_base.cc:1034] running on GCE node
I20260807 11:28:03.795922 3269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:03.796872 3269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:03.798115 3269 hybrid_clock.cc:648] HybridClock initialized: now 1786102083798057 us; error 46 us; skew 500 ppm
I20260807 11:28:03.798465 3269 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:03 dist-test-slave-2w3w krb5kdc[2875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102083, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:03.801775 3269 init.cc:377] Logged in from keytab as kudu/127.16.197.3@KRBTEST.COM (short username kudu)
I20260807 11:28:03.803380 3269 webserver.cc:460] Webserver started at http://127.16.197.3:34233/ using document root <none> and password file <none>
I20260807 11:28:03.804055 3269 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:03.804159 3269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:03.804515 3269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:03.808043 3269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-2/data/instance:
uuid: "47b445261bc342e4888e8250cde16547"
format_stamp: "Formatted at 2026-08-07 11:28:03 on dist-test-slave-2w3w"
server_key: "5b6683cf0b5dc91e8c4420ee337c378e"
server_key_iv: "79714ce343e66b343d78dab2e8c7062a"
server_key_version: "encryptionkey@0"
I20260807 11:28:03.808795 3269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance:
uuid: "47b445261bc342e4888e8250cde16547"
format_stamp: "Formatted at 2026-08-07 11:28:03 on dist-test-slave-2w3w"
server_key: "5b6683cf0b5dc91e8c4420ee337c378e"
server_key_iv: "79714ce343e66b343d78dab2e8c7062a"
server_key_version: "encryptionkey@0"
I20260807 11:28:03.814589 3269 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 11:28:03.819129 3284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:03.820644 3269 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 11:28:03.820778 3269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "47b445261bc342e4888e8250cde16547"
format_stamp: "Formatted at 2026-08-07 11:28:03 on dist-test-slave-2w3w"
server_key: "5b6683cf0b5dc91e8c4420ee337c378e"
server_key_iv: "79714ce343e66b343d78dab2e8c7062a"
server_key_version: "encryptionkey@0"
I20260807 11:28:03.820962 3269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:03.847096 3123 heartbeater.cc:503] Master 127.16.197.62:38263 was elected leader, sending a full tablet report...
I20260807 11:28:03.875084 3269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:03.879642 3269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:03.879917 3269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:03.880919 3269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:28:03.882244 3269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:28:03.882318 3269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:03.882402 3269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:28:03.882447 3269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:03.917834 3269 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:32885
I20260807 11:28:03.917920 3401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:32885 every 8 connection(s)
I20260807 11:28:03.919245 3269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:28:03.923877 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 3269
I20260807 11:28:03.924094 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance
I20260807 11:28:03.924733 17172 external_mini_cluster.cc:1428] Setting key 714ca9e52177e334a66e0ac419561da4
Aug 07 11:28:03 dist-test-slave-2w3w krb5kdc[2875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102083, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:28:03.940536 3404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:03.921897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:38241 (local address 127.16.197.62:38263)
0807 11:28:03.922362 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:03.922369 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:03.923444 (+ 1075us) server_negotiation.cc:407] Connection header received
0807 11:28:03.926054 (+ 2610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:03.926060 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:03.926206 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:03.926329 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:03.928386 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:03.929168 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:03.930677 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:03.931162 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:03.931261 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:03.934561 (+ 3300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:03.934566 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:03.934569 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:03.934603 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:03.937348 (+ 2745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:03.937995 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:03.937998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:03.938000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:03.938120 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:03.938524 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:03.938530 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:03.938533 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:03.938873 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:03.939053 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:03.939957 (+ 904us) server_negotiation.cc:299] Negotiation successful
0807 11:28:03.940147 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":213,"threads_started":1}
I20260807 11:28:03.941498 3406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:03.922486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:38263 (local address 127.16.197.3:38241)
0807 11:28:03.923266 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:03.923317 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:03.925758 (+ 2441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:03.926650 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:03.926662 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:03.927451 (+ 789us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:28:03.928211 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:03.928238 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:03.929301 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:03.929304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:03.930551 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:03.930563 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:03.931352 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:03.931356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:03.931564 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:03.932192 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:28:03.932214 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:28:03.934362 (+ 2148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:28:03.937466 (+ 3104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:28:03.937472 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:28:03.937483 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:28:03.937887 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:28:03.938211 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:28:03.938214 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:28:03.938216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:28:03.938403 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:28:03.939079 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:28:03.939087 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:28:03.939508 (+ 421us) client_negotiation.cc:769] Sending connection context
0807 11:28:03.939817 (+ 309us) client_negotiation.cc:240] Negotiation successful
0807 11:28:03.940369 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":274,"threads_started":1}
I20260807 11:28:03.942811 3402 heartbeater.cc:347] Connected to a master server at 127.16.197.62:38263
I20260807 11:28:03.943151 3402 heartbeater.cc:464] Registering TS with master...
I20260807 11:28:03.943848 3402 heartbeater.cc:511] Master 127.16.197.62:38263 requested a full tablet report, sending...
I20260807 11:28:03.945182 2923 ts_manager.cc:194] Registered new tserver with Master: 47b445261bc342e4888e8250cde16547 (127.16.197.3:32885)
I20260807 11:28:03.945957 2923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.3@KRBTEST.COM'} at 127.16.197.3:38241
I20260807 11:28:03.953065 17172 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 11:28:03.973487 3404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:03.958059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57438 (local address 127.16.197.62:38263)
0807 11:28:03.958275 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:03.958283 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:03.958705 (+ 422us) server_negotiation.cc:407] Connection header received
0807 11:28:03.959000 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:03.959007 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:03.959132 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:03.959230 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:03.961297 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:03.962079 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:03.963602 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:03.964020 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:03.964165 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:03.966791 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:03.966796 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:03.966799 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:03.966837 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:03.969912 (+ 3075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:03.970854 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:03.970860 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:03.970865 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:03.971106 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:03.971589 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:03.971598 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:03.971602 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:03.972106 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:03.972485 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:03.972877 (+ 392us) server_negotiation.cc:299] Negotiation successful
0807 11:28:03.973141 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 11:28:03.977998 2923 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57438:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 11:28:03.985055 2923 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:28:04.021555 3420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:03.998239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41028 (local address 127.16.197.1:34535)
0807 11:28:04.011481 (+ 13242us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:04.011492 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:04.011525 (+ 33us) server_negotiation.cc:407] Connection header received
0807 11:28:04.011631 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:04.011639 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:04.011937 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:04.012997 (+ 1060us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:04.013330 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:04.015612 (+ 2282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:04.019045 (+ 3433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:04.020373 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:04.020534 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:04.020793 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:04.021029 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 11:28:04.021201 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13050,"thread_start_us":12874,"threads_started":1}
I20260807 11:28:04.021659 3421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:03.997904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:34535 (local address 127.0.0.1:41028)
0807 11:28:04.010503 (+ 12599us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:04.010559 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:04.010766 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:04.012201 (+ 1435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:04.012209 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:04.012253 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:04.012694 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:04.012710 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:04.015825 (+ 3115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:04.015834 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:04.018837 (+ 3003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:04.018853 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:04.020568 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:04.020574 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:04.020766 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:04.020820 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 11:28:04.020984 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 11:28:04.021174 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12297,"mutex_wait_us":31,"thread_start_us":13653,"threads_started":1}
I20260807 11:28:04.022918 3422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:03.999061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52904 (local address 127.16.197.3:32885)
0807 11:28:04.010634 (+ 11573us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:04.010647 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:04.012183 (+ 1536us) server_negotiation.cc:407] Connection header received
0807 11:28:04.012295 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:04.012304 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:04.012521 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:04.012737 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:04.013989 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:04.016180 (+ 2191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:04.019961 (+ 3781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:04.021463 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:04.021593 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:04.021834 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:04.022403 (+ 569us) server_negotiation.cc:299] Negotiation successful
0807 11:28:04.022562 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11315,"thread_start_us":11087,"threads_started":1}
I20260807 11:28:04.023803 3419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:03.998268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:32885 (local address 127.0.0.1:52904)
0807 11:28:04.011870 (+ 13602us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:04.011901 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:04.012046 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:04.013136 (+ 1090us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:04.013143 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:04.013181 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:04.013717 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:04.013732 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:04.016386 (+ 2654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:04.016394 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:04.019794 (+ 3400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:04.019809 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:04.021969 (+ 2160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:04.021975 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:04.022159 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:04.022201 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 11:28:04.023191 (+ 990us) client_negotiation.cc:240] Negotiation successful
0807 11:28:04.023325 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13459,"mutex_wait_us":791,"thread_start_us":12166,"threads_started":1}
I20260807 11:28:04.027204 3423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:04.012490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:38477 (local address 127.0.0.1:39574)
0807 11:28:04.018155 (+ 5665us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:04.018175 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:04.018296 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:04.018937 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:04.018942 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:04.018997 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:04.019424 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:04.019435 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:04.021254 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:04.021260 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:04.024526 (+ 3266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:04.024553 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:04.026431 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:04.026437 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:04.026596 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:04.026628 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 11:28:04.026725 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 11:28:04.026863 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5526,"thread_start_us":5367,"threads_started":1}
I20260807 11:28:04.026681 3053 tablet_service.cc:1467] Processing CreateTablet for tablet b3bea59a2e354cf38a704ce3aba01d8c (DEFAULT_TABLE table=test-table [id=3de5902953464b1aa0c1cd4a97f94d5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:28:04.027357 3424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:04.012573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39574 (local address 127.16.197.2:38477)
0807 11:28:04.018464 (+ 5891us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:04.018472 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:04.018504 (+ 32us) server_negotiation.cc:407] Connection header received
0807 11:28:04.018585 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:04.018590 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:04.018741 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:04.018959 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:04.019595 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:04.021063 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:04.024695 (+ 3632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:04.026184 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:04.026335 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:04.026513 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:04.026825 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 11:28:04.027016 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5721,"thread_start_us":5581,"threads_started":1}
I20260807 11:28:04.027846 3334 tablet_service.cc:1467] Processing CreateTablet for tablet b3bea59a2e354cf38a704ce3aba01d8c (DEFAULT_TABLE table=test-table [id=3de5902953464b1aa0c1cd4a97f94d5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:28:04.028370 3053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3bea59a2e354cf38a704ce3aba01d8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:04.029263 3334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3bea59a2e354cf38a704ce3aba01d8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:04.031378 3194 tablet_service.cc:1467] Processing CreateTablet for tablet b3bea59a2e354cf38a704ce3aba01d8c (DEFAULT_TABLE table=test-table [id=3de5902953464b1aa0c1cd4a97f94d5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:28:04.032732 3194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3bea59a2e354cf38a704ce3aba01d8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:04.040606 3425 tablet_bootstrap.cc:492] T b3bea59a2e354cf38a704ce3aba01d8c P 226ded7d1c91473ebd5aad855f22ea37: Bootstrap starting.
I20260807 11:28:04.043896 3426 tablet_bootstrap.cc:492] T b3bea59a2e354cf38a704ce3aba01d8c P 47b445261bc342e4888e8250cde16547: Bootstrap starting.
I20260807 11:28:04.044325 3425 tablet_bootstrap.cc:654] T b3bea59a2e354cf38a704ce3aba01d8c P 226ded7d1c91473ebd5aad855f22ea37: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:04.045961 3425 log.cc:824] T b3bea59a2e354cf38a704ce3aba01d8c P 226ded7d1c91473ebd5aad855f22ea37: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:04.046859 3427 tablet_bootstrap.cc:492] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a: Bootstrap starting.
I20260807 11:28:04.047849 3426 tablet_bootstrap.cc:654] T b3bea59a2e354cf38a704ce3aba01d8c P 47b445261bc342e4888e8250cde16547: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:04.049384 3426 log.cc:824] T b3bea59a2e354cf38a704ce3aba01d8c P 47b445261bc342e4888e8250cde16547: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:04.050186 3427 tablet_bootstrap.cc:654] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:04.051234 3427 log.cc:824] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:04.054144 3427 tablet_bootstrap.cc:492] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a: No bootstrap required, opened a new log
I20260807 11:28:04.054427 3426 tablet_bootstrap.cc:492] T b3bea59a2e354cf38a704ce3aba01d8c P 47b445261bc342e4888e8250cde16547: No bootstrap required, opened a new log
I20260807 11:28:04.054463 3427 ts_tablet_manager.cc:1397] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260807 11:28:04.054687 3426 ts_tablet_manager.cc:1397] T b3bea59a2e354cf38a704ce3aba01d8c P 47b445261bc342e4888e8250cde16547: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20260807 11:28:04.057143 3425 tablet_bootstrap.cc:492] T b3bea59a2e354cf38a704ce3aba01d8c P 226ded7d1c91473ebd5aad855f22ea37: No bootstrap required, opened a new log
I20260807 11:28:04.057466 3425 ts_tablet_manager.cc:1397] T b3bea59a2e354cf38a704ce3aba01d8c P 226ded7d1c91473ebd5aad855f22ea37: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.008s
I20260807 11:28:04.062248 3426 raft_consensus.cc:348] T b3bea59a2e354cf38a704ce3aba01d8c P 47b445261bc342e4888e8250cde16547 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "226ded7d1c91473ebd5aad855f22ea37" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 34535 } } peers { permanent_uuid: "47b445261bc342e4888e8250cde16547" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 32885 } } peers { permanent_uuid: "e8fa390f893444618de726274a38864a" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38477 } }
I20260807 11:28:04.062639 3426 raft_consensus.cc:374] T b3bea59a2e354cf38a704ce3aba01d8c P 47b445261bc342e4888e8250cde16547 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:04.062743 3426 raft_consensus.cc:729] T b3bea59a2e354cf38a704ce3aba01d8c P 47b445261bc342e4888e8250cde16547 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47b445261bc342e4888e8250cde16547, State: Initialized, Role: FOLLOWER
I20260807 11:28:04.063544 3426 consensus_queue.cc:260] T b3bea59a2e354cf38a704ce3aba01d8c P 47b445261bc342e4888e8250cde16547 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "226ded7d1c91473ebd5aad855f22ea37" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 34535 } } peers { permanent_uuid: "47b445261bc342e4888e8250cde16547" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 32885 } } peers { permanent_uuid: "e8fa390f893444618de726274a38864a" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38477 } }
I20260807 11:28:04.064711 3402 heartbeater.cc:503] Master 127.16.197.62:38263 was elected leader, sending a full tablet report...
I20260807 11:28:04.065448 3426 ts_tablet_manager.cc:1428] T b3bea59a2e354cf38a704ce3aba01d8c P 47b445261bc342e4888e8250cde16547: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260807 11:28:04.065407 3427 raft_consensus.cc:348] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "226ded7d1c91473ebd5aad855f22ea37" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 34535 } } peers { permanent_uuid: "47b445261bc342e4888e8250cde16547" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 32885 } } peers { permanent_uuid: "e8fa390f893444618de726274a38864a" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38477 } }
I20260807 11:28:04.065857 3427 raft_consensus.cc:374] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:04.065968 3427 raft_consensus.cc:729] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8fa390f893444618de726274a38864a, State: Initialized, Role: FOLLOWER
I20260807 11:28:04.066555 3427 consensus_queue.cc:260] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "226ded7d1c91473ebd5aad855f22ea37" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 34535 } } peers { permanent_uuid: "47b445261bc342e4888e8250cde16547" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 32885 } } peers { permanent_uuid: "e8fa390f893444618de726274a38864a" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38477 } }
I20260807 11:28:04.067689 3263 heartbeater.cc:503] Master 127.16.197.62:38263 was elected leader, sending a full tablet report...
I20260807 11:28:04.069569 3427 ts_tablet_manager.cc:1428] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260807 11:28:04.069288 3425 raft_consensus.cc:348] T b3bea59a2e354cf38a704ce3aba01d8c P 226ded7d1c91473ebd5aad855f22ea37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "226ded7d1c91473ebd5aad855f22ea37" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 34535 } } peers { permanent_uuid: "47b445261bc342e4888e8250cde16547" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 32885 } } peers { permanent_uuid: "e8fa390f893444618de726274a38864a" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38477 } }
I20260807 11:28:04.069734 3425 raft_consensus.cc:374] T b3bea59a2e354cf38a704ce3aba01d8c P 226ded7d1c91473ebd5aad855f22ea37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:04.069851 3425 raft_consensus.cc:729] T b3bea59a2e354cf38a704ce3aba01d8c P 226ded7d1c91473ebd5aad855f22ea37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 226ded7d1c91473ebd5aad855f22ea37, State: Initialized, Role: FOLLOWER
I20260807 11:28:04.070482 3425 consensus_queue.cc:260] T b3bea59a2e354cf38a704ce3aba01d8c P 226ded7d1c91473ebd5aad855f22ea37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "226ded7d1c91473ebd5aad855f22ea37" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 34535 } } peers { permanent_uuid: "47b445261bc342e4888e8250cde16547" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 32885 } } peers { permanent_uuid: "e8fa390f893444618de726274a38864a" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38477 } }
I20260807 11:28:04.072796 3425 ts_tablet_manager.cc:1428] T b3bea59a2e354cf38a704ce3aba01d8c P 226ded7d1c91473ebd5aad855f22ea37: Time spent starting tablet: real 0.015s user 0.011s sys 0.003s
W20260807 11:28:04.092113 3264 tablet.cc:2369] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:28:04.146413 3432 raft_consensus.cc:482] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:28:04.146807 3432 raft_consensus.cc:504] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "226ded7d1c91473ebd5aad855f22ea37" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 34535 } } peers { permanent_uuid: "47b445261bc342e4888e8250cde16547" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 32885 } } peers { permanent_uuid: "e8fa390f893444618de726274a38864a" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38477 } }
I20260807 11:28:04.148706 3432 leader_election.cc:288] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 226ded7d1c91473ebd5aad855f22ea37 (127.16.197.1:34535), 47b445261bc342e4888e8250cde16547 (127.16.197.3:32885)
I20260807 11:28:04.156181 3420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:04.149136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:33011 (local address 127.16.197.1:34535)
0807 11:28:04.149410 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:04.149419 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:04.150200 (+ 781us) server_negotiation.cc:407] Connection header received
0807 11:28:04.150456 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:04.150460 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:04.150533 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:04.150633 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:04.151312 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:04.152031 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:04.153997 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:04.154895 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:04.155106 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:04.155189 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:04.155753 (+ 564us) server_negotiation.cc:299] Negotiation successful
0807 11:28:04.155904 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 11:28:04.156190 3422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:04.149339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:47981 (local address 127.16.197.3:32885)
0807 11:28:04.149614 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:04.149635 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:04.150398 (+ 763us) server_negotiation.cc:407] Connection header received
0807 11:28:04.150609 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:04.150613 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:04.150671 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:04.150774 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:04.151440 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:04.152159 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:04.154000 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:04.155041 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:04.155208 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:04.155280 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:04.155813 (+ 533us) server_negotiation.cc:299] Negotiation successful
0807 11:28:04.155951 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260807 11:28:04.157018 3435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:04.149119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:32885 (local address 127.16.197.2:47981)
0807 11:28:04.150278 (+ 1159us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:04.150292 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:04.150459 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:04.150795 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:04.150799 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:04.150834 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:04.151280 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:04.151288 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:04.152309 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:04.152314 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:04.153834 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:04.153847 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:04.155359 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:04.155365 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:04.155590 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:04.155636 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 11:28:04.156437 (+ 801us) client_negotiation.cc:240] Negotiation successful
0807 11:28:04.156590 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1022,"mutex_wait_us":27,"thread_start_us":809,"threads_started":1}
I20260807 11:28:04.157603 3434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:04.149034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:34535 (local address 127.16.197.2:33011)
0807 11:28:04.150066 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:04.150083 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:04.150288 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:04.150656 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:04.150660 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:04.150687 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:04.151139 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:04.151150 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:04.152165 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:04.152172 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:04.153836 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:04.153848 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:04.155230 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:04.155238 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:04.155473 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:04.155561 (+ 88us) client_negotiation.cc:769] Sending connection context
0807 11:28:04.156721 (+ 1160us) client_negotiation.cc:240] Negotiation successful
0807 11:28:04.156868 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":908,"mutex_wait_us":41,"thread_start_us":606,"threads_started":1}
I20260807 11:28:04.158141 3355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3bea59a2e354cf38a704ce3aba01d8c" candidate_uuid: "e8fa390f893444618de726274a38864a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47b445261bc342e4888e8250cde16547" is_pre_election: true
I20260807 11:28:04.158569 3074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3bea59a2e354cf38a704ce3aba01d8c" candidate_uuid: "e8fa390f893444618de726274a38864a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "226ded7d1c91473ebd5aad855f22ea37" is_pre_election: true
I20260807 11:28:04.158744 3355 raft_consensus.cc:2454] T b3bea59a2e354cf38a704ce3aba01d8c P 47b445261bc342e4888e8250cde16547 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8fa390f893444618de726274a38864a in term 0.
I20260807 11:28:04.159221 3074 raft_consensus.cc:2454] T b3bea59a2e354cf38a704ce3aba01d8c P 226ded7d1c91473ebd5aad855f22ea37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8fa390f893444618de726274a38864a in term 0.
I20260807 11:28:04.159544 3145 leader_election.cc:302] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47b445261bc342e4888e8250cde16547, e8fa390f893444618de726274a38864a; no voters:
I20260807 11:28:04.160010 3432 raft_consensus.cc:2781] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:28:04.160167 3432 raft_consensus.cc:482] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:28:04.160260 3432 raft_consensus.cc:3037] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:04.163511 3432 raft_consensus.cc:504] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "226ded7d1c91473ebd5aad855f22ea37" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 34535 } } peers { permanent_uuid: "47b445261bc342e4888e8250cde16547" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 32885 } } peers { permanent_uuid: "e8fa390f893444618de726274a38864a" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38477 } }
I20260807 11:28:04.164243 3432 leader_election.cc:288] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a [CANDIDATE]: Term 1 election: Requested vote from peers 226ded7d1c91473ebd5aad855f22ea37 (127.16.197.1:34535), 47b445261bc342e4888e8250cde16547 (127.16.197.3:32885)
I20260807 11:28:04.164729 3355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3bea59a2e354cf38a704ce3aba01d8c" candidate_uuid: "e8fa390f893444618de726274a38864a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47b445261bc342e4888e8250cde16547"
I20260807 11:28:04.164739 3074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3bea59a2e354cf38a704ce3aba01d8c" candidate_uuid: "e8fa390f893444618de726274a38864a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "226ded7d1c91473ebd5aad855f22ea37"
I20260807 11:28:04.164944 3355 raft_consensus.cc:3037] T b3bea59a2e354cf38a704ce3aba01d8c P 47b445261bc342e4888e8250cde16547 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:04.164940 3074 raft_consensus.cc:3037] T b3bea59a2e354cf38a704ce3aba01d8c P 226ded7d1c91473ebd5aad855f22ea37 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:04.167367 3355 raft_consensus.cc:2454] T b3bea59a2e354cf38a704ce3aba01d8c P 47b445261bc342e4888e8250cde16547 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8fa390f893444618de726274a38864a in term 1.
I20260807 11:28:04.167804 3145 leader_election.cc:302] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47b445261bc342e4888e8250cde16547, e8fa390f893444618de726274a38864a; no voters:
I20260807 11:28:04.168088 3432 raft_consensus.cc:2781] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:04.168162 3074 raft_consensus.cc:2454] T b3bea59a2e354cf38a704ce3aba01d8c P 226ded7d1c91473ebd5aad855f22ea37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8fa390f893444618de726274a38864a in term 1.
W20260807 11:28:04.170939 3403 tablet.cc:2369] T b3bea59a2e354cf38a704ce3aba01d8c P 47b445261bc342e4888e8250cde16547: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:28:04.171255 3432 raft_consensus.cc:686] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a [term 1 LEADER]: Becoming Leader. State: Replica: e8fa390f893444618de726274a38864a, State: Running, Role: LEADER
I20260807 11:28:04.171844 3432 consensus_queue.cc:237] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "226ded7d1c91473ebd5aad855f22ea37" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 34535 } } peers { permanent_uuid: "47b445261bc342e4888e8250cde16547" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 32885 } } peers { permanent_uuid: "e8fa390f893444618de726274a38864a" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38477 } }
I20260807 11:28:04.176388 2923 catalog_manager.cc:5495] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a reported cstate change: term changed from 0 to 1, leader changed from <none> to e8fa390f893444618de726274a38864a (127.16.197.2). New cstate: current_term: 1 leader_uuid: "e8fa390f893444618de726274a38864a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "226ded7d1c91473ebd5aad855f22ea37" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 34535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47b445261bc342e4888e8250cde16547" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 32885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8fa390f893444618de726274a38864a" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38477 } health_report { overall_health: HEALTHY } } }
I20260807 11:28:04.206405 3424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:04.200731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39580 (local address 127.16.197.2:38477)
0807 11:28:04.200978 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:04.200987 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:04.201009 (+ 22us) server_negotiation.cc:407] Connection header received
0807 11:28:04.201133 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:04.201137 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:04.201214 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:04.201373 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:28:04.201836 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:04.202581 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:04.204200 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:04.204613 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:04.204760 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:04.205269 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:28:04.205788 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:28:04.205968 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:04.206027 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 11:28:04.206114 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 11:28:04.215263 3355 raft_consensus.cc:1261] T b3bea59a2e354cf38a704ce3aba01d8c P 47b445261bc342e4888e8250cde16547 [term 1 FOLLOWER]: Refusing update from remote peer e8fa390f893444618de726274a38864a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:28:04.215314 3074 raft_consensus.cc:1261] T b3bea59a2e354cf38a704ce3aba01d8c P 226ded7d1c91473ebd5aad855f22ea37 [term 1 FOLLOWER]: Refusing update from remote peer e8fa390f893444618de726274a38864a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:28:04.216346 3436 consensus_queue.cc:1035] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a [LEADER]: Connected to new peer: Peer: permanent_uuid: "226ded7d1c91473ebd5aad855f22ea37" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 34535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:28:04.216740 3432 consensus_queue.cc:1035] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a [LEADER]: Connected to new peer: Peer: permanent_uuid: "47b445261bc342e4888e8250cde16547" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 32885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:28:04.229161 3441 mvcc.cc:205] Tried to move back new op lower bound from 7315874136933642240 to 7315874136771928064. Current Snapshot: MvccSnapshot[applied={T|T < 7315874136933642240}]
I20260807 11:28:04.232184 3440 mvcc.cc:205] Tried to move back new op lower bound from 7315874136933642240 to 7315874136771928064. Current Snapshot: MvccSnapshot[applied={T|T < 7315874136933642240 or (T in {7315874136933642240})}]
I20260807 11:28:04.242676 2923 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:28:04.243034 2923 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:28:04.246991 2923 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a9a6789029734079944ec94637f552b4: Sending DeleteTablet for 3 replicas of tablet b3bea59a2e354cf38a704ce3aba01d8c
I20260807 11:28:04.248379 3053 tablet_service.cc:1514] Processing DeleteTablet for tablet b3bea59a2e354cf38a704ce3aba01d8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:28:04 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:41028
I20260807 11:28:04.248317 3334 tablet_service.cc:1514] Processing DeleteTablet for tablet b3bea59a2e354cf38a704ce3aba01d8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:28:04 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:52904
I20260807 11:28:04.248859 3194 tablet_service.cc:1514] Processing DeleteTablet for tablet b3bea59a2e354cf38a704ce3aba01d8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:28:04 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:39574
I20260807 11:28:04.251094 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 2987
I20260807 11:28:04.264904 3450 tablet_replica.cc:330] T b3bea59a2e354cf38a704ce3aba01d8c P 226ded7d1c91473ebd5aad855f22ea37: stopping tablet replica
I20260807 11:28:04.265588 3450 raft_consensus.cc:2229] T b3bea59a2e354cf38a704ce3aba01d8c P 226ded7d1c91473ebd5aad855f22ea37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:28:04.266327 3450 raft_consensus.cc:2258] T b3bea59a2e354cf38a704ce3aba01d8c P 226ded7d1c91473ebd5aad855f22ea37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:28:04.267379 3451 tablet_replica.cc:330] T b3bea59a2e354cf38a704ce3aba01d8c P 47b445261bc342e4888e8250cde16547: stopping tablet replica
I20260807 11:28:04.267908 3451 raft_consensus.cc:2229] T b3bea59a2e354cf38a704ce3aba01d8c P 47b445261bc342e4888e8250cde16547 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:28:04.268592 3452 tablet_replica.cc:330] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a: stopping tablet replica
I20260807 11:28:04.268970 3451 raft_consensus.cc:2258] T b3bea59a2e354cf38a704ce3aba01d8c P 47b445261bc342e4888e8250cde16547 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:28:04.269013 3452 raft_consensus.cc:2229] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a [term 1 LEADER]: Raft consensus shutting down.
I20260807 11:28:04.269704 3452 raft_consensus.cc:2258] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:28:04.271796 3450 ts_tablet_manager.cc:1905] T b3bea59a2e354cf38a704ce3aba01d8c P 226ded7d1c91473ebd5aad855f22ea37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:28:04.271796 3451 ts_tablet_manager.cc:1905] T b3bea59a2e354cf38a704ce3aba01d8c P 47b445261bc342e4888e8250cde16547: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:28:04.272261 3452 ts_tablet_manager.cc:1905] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:28:04.279075 3450 ts_tablet_manager.cc:1918] T b3bea59a2e354cf38a704ce3aba01d8c P 226ded7d1c91473ebd5aad855f22ea37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:28:04.279255 3450 log.cc:1196] T b3bea59a2e354cf38a704ce3aba01d8c P 226ded7d1c91473ebd5aad855f22ea37: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-0/wal/wals/b3bea59a2e354cf38a704ce3aba01d8c
I20260807 11:28:04.279839 3450 ts_tablet_manager.cc:1939] T b3bea59a2e354cf38a704ce3aba01d8c P 226ded7d1c91473ebd5aad855f22ea37: Deleting consensus metadata
I20260807 11:28:04.280203 3452 ts_tablet_manager.cc:1918] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:28:04.280474 3452 log.cc:1196] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-1/wal/wals/b3bea59a2e354cf38a704ce3aba01d8c
I20260807 11:28:04.281131 3452 ts_tablet_manager.cc:1939] T b3bea59a2e354cf38a704ce3aba01d8c P e8fa390f893444618de726274a38864a: Deleting consensus metadata
I20260807 11:28:04.281981 2908 catalog_manager.cc:4841] TS 226ded7d1c91473ebd5aad855f22ea37 (127.16.197.1:34535): tablet b3bea59a2e354cf38a704ce3aba01d8c (table test-table [id=3de5902953464b1aa0c1cd4a97f94d5a]) successfully deleted
I20260807 11:28:04.283017 2908 catalog_manager.cc:4841] TS e8fa390f893444618de726274a38864a (127.16.197.2:38477): tablet b3bea59a2e354cf38a704ce3aba01d8c (table test-table [id=3de5902953464b1aa0c1cd4a97f94d5a]) successfully deleted
Aug 07 11:28:04 dist-test-slave-2w3w krb5kdc[2875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.1@KRBTEST.COM
I20260807 11:28:04.285498 3451 ts_tablet_manager.cc:1918] T b3bea59a2e354cf38a704ce3aba01d8c P 47b445261bc342e4888e8250cde16547: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:28:04.285669 3451 log.cc:1196] T b3bea59a2e354cf38a704ce3aba01d8c P 47b445261bc342e4888e8250cde16547: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786101765888975-17172-0/minicluster-data/ts-2/wal/wals/b3bea59a2e354cf38a704ce3aba01d8c
I20260807 11:28:04.286242 3451 ts_tablet_manager.cc:1939] T b3bea59a2e354cf38a704ce3aba01d8c P 47b445261bc342e4888e8250cde16547: Deleting consensus metadata
I20260807 11:28:04.288206 2906 catalog_manager.cc:4841] TS 47b445261bc342e4888e8250cde16547 (127.16.197.3:32885): tablet b3bea59a2e354cf38a704ce3aba01d8c (table test-table [id=3de5902953464b1aa0c1cd4a97f94d5a]) successfully deleted
I20260807 11:28:04.293190 3420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:04.252773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41036 (local address 127.16.197.1:34535)
0807 11:28:04.253026 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:04.253036 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:04.267776 (+ 14740us) server_negotiation.cc:407] Connection header received
0807 11:28:04.268010 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:04.268017 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:04.268128 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:04.268241 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:04.272152 (+ 3911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:04.273342 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:04.279095 (+ 5753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:04.279796 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:04.279928 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:04.286577 (+ 6649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:04.286593 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:04.286613 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:04.286656 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:04.290084 (+ 3428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:04.291084 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:04.291091 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:04.291094 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:04.291237 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:04.291671 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:04.291675 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:04.291677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:04.292036 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:04.292261 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:04.292637 (+ 376us) server_negotiation.cc:299] Negotiation successful
0807 11:28:04.292805 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 11:28:04.293905 3118 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:04.400703 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 2987
I20260807 11:28:04.427886 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 3129
Aug 07 11:28:04 dist-test-slave-2w3w krb5kdc[2875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.2@KRBTEST.COM
I20260807 11:28:04.446190 3424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:04.428823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39592 (local address 127.16.197.2:38477)
0807 11:28:04.429038 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:04.429044 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:04.429067 (+ 23us) server_negotiation.cc:407] Connection header received
0807 11:28:04.429173 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:04.429179 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:04.429267 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:04.429404 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:04.430688 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:04.432084 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:04.433361 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:04.434079 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:04.434210 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:04.438848 (+ 4638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:04.438866 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:04.438888 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:04.438950 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:04.443206 (+ 4256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:04.443968 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:04.443979 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:04.443989 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:04.444179 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:04.444549 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:04.444554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:04.444558 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:04.445042 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:04.445375 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:04.445530 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 11:28:04.445777 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 11:28:04.446924 3258 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:04.563066 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 3129
I20260807 11:28:04.584100 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 3269
Aug 07 11:28:04 dist-test-slave-2w3w krb5kdc[2875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.3@KRBTEST.COM
I20260807 11:28:04.599205 3422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:04.584908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52918 (local address 127.16.197.3:32885)
0807 11:28:04.585093 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:04.585098 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:04.585117 (+ 19us) server_negotiation.cc:407] Connection header received
0807 11:28:04.585179 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:04.585183 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:04.585248 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:04.585340 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:04.586607 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:04.587415 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:04.588743 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:04.589109 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:04.589197 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:04.593626 (+ 4429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:04.593639 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:04.593655 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:04.593689 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:04.596565 (+ 2876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:04.597260 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:04.597266 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:04.597268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:04.597446 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:04.597788 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:04.597791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:04.597793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:04.598120 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:04.598333 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:04.598608 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 11:28:04.598789 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 11:28:04.599969 3397 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:04.690150 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 3269
I20260807 11:28:04.711952 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 2889
I20260807 11:28:04.713177 2954 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:04.826282 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 2889
2026-08-07T11:28:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4580 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T11:28:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:28:04Z Disabled control of system clock
I20260807 11:28:04.916265 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:33133
--webserver_interface=127.16.197.62
--webserver_port=0
--builtin_ntp_servers=127.16.197.20:44519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:33133
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 11:28:05.263528 3461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:05.263875 3461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:05.264051 3461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:05.271318 3461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:28:05.271450 3461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:05.271530 3461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:05.271629 3461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:28:05.271710 3461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:28:05.280829 3461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:44519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:33133
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:33133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:05.282188 3461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:05.284047 3461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:05.296190 3467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:05.296343 3466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:05.296270 3469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:05.297235 3461 server_base.cc:1034] running on GCE node
I20260807 11:28:05.298084 3461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:05.299564 3461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:05.300799 3461 hybrid_clock.cc:648] HybridClock initialized: now 1786102085300764 us; error 51 us; skew 500 ppm
I20260807 11:28:05.301144 3461 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 11:28:05.307240 3461 webserver.cc:460] Webserver started at http://127.16.197.62:39899/ using document root <none> and password file <none>
I20260807 11:28:05.307962 3461 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:05.308101 3461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:05.308472 3461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:05.311481 3461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "405440a089db4eb09b8d82350eb2e9a1"
format_stamp: "Formatted at 2026-08-07 11:28:05 on dist-test-slave-2w3w"
server_key: "80a000ca7297c34728731ff4c25cf557"
server_key_iv: "064f8f4e16ab330829fa33380303c695"
server_key_version: "encryptionkey@0"
I20260807 11:28:05.312376 3461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "405440a089db4eb09b8d82350eb2e9a1"
format_stamp: "Formatted at 2026-08-07 11:28:05 on dist-test-slave-2w3w"
server_key: "80a000ca7297c34728731ff4c25cf557"
server_key_iv: "064f8f4e16ab330829fa33380303c695"
server_key_version: "encryptionkey@0"
I20260807 11:28:05.317610 3461 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 11:28:05.320979 3476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:05.322285 3461 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 11:28:05.322511 3461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "405440a089db4eb09b8d82350eb2e9a1"
format_stamp: "Formatted at 2026-08-07 11:28:05 on dist-test-slave-2w3w"
server_key: "80a000ca7297c34728731ff4c25cf557"
server_key_iv: "064f8f4e16ab330829fa33380303c695"
server_key_version: "encryptionkey@0"
I20260807 11:28:05.322734 3461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:05.340437 3461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:05.341275 3461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:05.341578 3461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:05.360891 3461 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:33133
I20260807 11:28:05.360941 3529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:33133 every 8 connection(s)
I20260807 11:28:05.362218 3461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:28:05.363436 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 3461
I20260807 11:28:05.363653 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:28:05.364288 17172 external_mini_cluster.cc:1428] Setting key aa8a2ae058bde96d025935dee876df7d
I20260807 11:28:05.367277 3530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:05.378253 3533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:05.366445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36884 (local address 127.16.197.62:33133)
0807 11:28:05.367610 (+ 1165us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:05.367621 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:05.367663 (+ 42us) server_negotiation.cc:407] Connection header received
0807 11:28:05.369246 (+ 1583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:05.369267 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:05.369692 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:05.370109 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:05.370691 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:05.372097 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:05.373654 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:05.374329 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:05.374541 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:05.375929 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:05.375937 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:05.375953 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:05.376530 (+ 577us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:05.376610 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:05.376634 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:05.376766 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:05.377009 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 11:28:05.377332 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":721,"thread_start_us":504,"threads_started":1}
I20260807 11:28:05.379803 3530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 405440a089db4eb09b8d82350eb2e9a1: Bootstrap starting.
I20260807 11:28:05.383332 3530 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 405440a089db4eb09b8d82350eb2e9a1: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:05.384475 3530 log.cc:824] T 00000000000000000000000000000000 P 405440a089db4eb09b8d82350eb2e9a1: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:05.387578 3530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 405440a089db4eb09b8d82350eb2e9a1: No bootstrap required, opened a new log
I20260807 11:28:05.394824 3530 raft_consensus.cc:348] T 00000000000000000000000000000000 P 405440a089db4eb09b8d82350eb2e9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "405440a089db4eb09b8d82350eb2e9a1" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33133 } }
I20260807 11:28:05.395277 3530 raft_consensus.cc:374] T 00000000000000000000000000000000 P 405440a089db4eb09b8d82350eb2e9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:05.395402 3530 raft_consensus.cc:729] T 00000000000000000000000000000000 P 405440a089db4eb09b8d82350eb2e9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 405440a089db4eb09b8d82350eb2e9a1, State: Initialized, Role: FOLLOWER
I20260807 11:28:05.395985 3530 consensus_queue.cc:260] T 00000000000000000000000000000000 P 405440a089db4eb09b8d82350eb2e9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "405440a089db4eb09b8d82350eb2e9a1" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33133 } }
I20260807 11:28:05.396216 3530 raft_consensus.cc:388] T 00000000000000000000000000000000 P 405440a089db4eb09b8d82350eb2e9a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:28:05.396344 3530 raft_consensus.cc:482] T 00000000000000000000000000000000 P 405440a089db4eb09b8d82350eb2e9a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:28:05.396529 3530 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 405440a089db4eb09b8d82350eb2e9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:05.398865 3530 raft_consensus.cc:504] T 00000000000000000000000000000000 P 405440a089db4eb09b8d82350eb2e9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "405440a089db4eb09b8d82350eb2e9a1" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33133 } }
I20260807 11:28:05.399438 3530 leader_election.cc:302] T 00000000000000000000000000000000 P 405440a089db4eb09b8d82350eb2e9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 405440a089db4eb09b8d82350eb2e9a1; no voters:
I20260807 11:28:05.400028 3530 leader_election.cc:288] T 00000000000000000000000000000000 P 405440a089db4eb09b8d82350eb2e9a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:28:05.400213 3535 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 405440a089db4eb09b8d82350eb2e9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:05.401829 3530 sys_catalog.cc:564] T 00000000000000000000000000000000 P 405440a089db4eb09b8d82350eb2e9a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:28:05.403452 3535 raft_consensus.cc:686] T 00000000000000000000000000000000 P 405440a089db4eb09b8d82350eb2e9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 405440a089db4eb09b8d82350eb2e9a1, State: Running, Role: LEADER
I20260807 11:28:05.404110 3535 consensus_queue.cc:237] T 00000000000000000000000000000000 P 405440a089db4eb09b8d82350eb2e9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "405440a089db4eb09b8d82350eb2e9a1" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33133 } }
I20260807 11:28:05.407577 3537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 405440a089db4eb09b8d82350eb2e9a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "405440a089db4eb09b8d82350eb2e9a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "405440a089db4eb09b8d82350eb2e9a1" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33133 } } }
I20260807 11:28:05.407567 3538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 405440a089db4eb09b8d82350eb2e9a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 405440a089db4eb09b8d82350eb2e9a1. Latest consensus state: current_term: 1 leader_uuid: "405440a089db4eb09b8d82350eb2e9a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "405440a089db4eb09b8d82350eb2e9a1" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33133 } } }
I20260807 11:28:05.408036 3537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 405440a089db4eb09b8d82350eb2e9a1 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:05.408080 3538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 405440a089db4eb09b8d82350eb2e9a1 [sys.catalog]: This master's current role is: LEADER
W20260807 11:28:05.412655 3551 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 405440a089db4eb09b8d82350eb2e9a1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 11:28:05.412789 3551 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 11:28:05.412954 3552 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:28:05.414772 3552 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:28:05.422977 3552 catalog_manager.cc:1347] Generated new cluster ID: fa2c83c4ef6448e48ae1529b7dfbdb0f
I20260807 11:28:05.423089 3552 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:28:05.440457 3552 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:28:05.441776 3552 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:28:05.448570 3552 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 405440a089db4eb09b8d82350eb2e9a1: Generated new TSK 0
I20260807 11:28:05.449412 3552 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:28:05.464074 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:0
--local_ip_for_outbound_sockets=127.16.197.1
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:33133
--builtin_ntp_servers=127.16.197.20:44519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 11:28:05.811281 3556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:05.811579 3556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:05.811721 3556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:05.819298 3556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:05.819404 3556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:05.819579 3556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:28:05.829460 3556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:44519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:33133
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:05.830869 3556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:05.832887 3556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:05.846931 3562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:05.846895 3564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:05.846896 3561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:05.848819 3556 server_base.cc:1034] running on GCE node
I20260807 11:28:05.849403 3556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:05.850318 3556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:05.851533 3556 hybrid_clock.cc:648] HybridClock initialized: now 1786102085851498 us; error 29 us; skew 500 ppm
I20260807 11:28:05.851890 3556 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 11:28:05.854193 3556 webserver.cc:460] Webserver started at http://127.16.197.1:42933/ using document root <none> and password file <none>
I20260807 11:28:05.854923 3556 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:05.855083 3556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:05.855412 3556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:05.858311 3556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-0/data/instance:
uuid: "2d40f11232d84fcd99be03bb8c4439b8"
format_stamp: "Formatted at 2026-08-07 11:28:05 on dist-test-slave-2w3w"
server_key: "15f232f26398825f9873256e36fc53e5"
server_key_iv: "7fea399a564d44275654e6757a7046fa"
server_key_version: "encryptionkey@0"
I20260807 11:28:05.859206 3556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance:
uuid: "2d40f11232d84fcd99be03bb8c4439b8"
format_stamp: "Formatted at 2026-08-07 11:28:05 on dist-test-slave-2w3w"
server_key: "15f232f26398825f9873256e36fc53e5"
server_key_iv: "7fea399a564d44275654e6757a7046fa"
server_key_version: "encryptionkey@0"
I20260807 11:28:05.864346 3556 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260807 11:28:05.867867 3570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:05.869211 3556 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 11:28:05.869390 3556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "2d40f11232d84fcd99be03bb8c4439b8"
format_stamp: "Formatted at 2026-08-07 11:28:05 on dist-test-slave-2w3w"
server_key: "15f232f26398825f9873256e36fc53e5"
server_key_iv: "7fea399a564d44275654e6757a7046fa"
server_key_version: "encryptionkey@0"
I20260807 11:28:05.869618 3556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:05.901938 3556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:05.902894 3556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:05.903262 3556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:05.904516 3556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:28:05.906255 3556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:28:05.906378 3556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:05.906498 3556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:28:05.906590 3556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:05.940984 3556 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:34965
I20260807 11:28:05.941088 3688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:34965 every 8 connection(s)
I20260807 11:28:05.942373 3556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:28:05.945868 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 3556
I20260807 11:28:05.946084 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance
I20260807 11:28:05.946642 17172 external_mini_cluster.cc:1428] Setting key 3fd818d849b2a875b2590f441cd679cf
I20260807 11:28:05.950100 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:0
--local_ip_for_outbound_sockets=127.16.197.2
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:33133
--builtin_ntp_servers=127.16.197.20:44519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 11:28:05.957064 3691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:05.945019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:43021 (local address 127.16.197.62:33133)
0807 11:28:05.945494 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:05.945501 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:05.946994 (+ 1493us) server_negotiation.cc:407] Connection header received
0807 11:28:05.949142 (+ 2148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:05.949146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:05.949215 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:05.949304 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:05.950755 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:05.951523 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:05.954139 (+ 2616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:05.954532 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:05.954629 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:05.956303 (+ 1674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:05.956308 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:05.956311 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:05.956335 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:05.956377 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:05.956389 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:05.956454 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:05.956723 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 11:28:05.956803 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":216,"threads_started":1}
I20260807 11:28:05.958572 3692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:05.945670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33133 (local address 127.16.197.1:43021)
0807 11:28:05.946768 (+ 1098us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:05.946824 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:05.948844 (+ 2020us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:05.949593 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:05.949604 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:05.949721 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:28:05.950595 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:05.950624 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:05.951719 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:05.951727 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:05.953970 (+ 2243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:05.953994 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:05.954780 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:05.954789 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:05.955145 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:05.955927 (+ 782us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 11:28:05.955933 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:28:05.955986 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 11:28:05.956088 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:28:05.956487 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:28:05.956500 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:28:05.956511 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 11:28:05.956703 (+ 192us) client_negotiation.cc:240] Negotiation successful
0807 11:28:05.957123 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":345,"threads_started":1}
I20260807 11:28:05.960215 3689 heartbeater.cc:347] Connected to a master server at 127.16.197.62:33133
I20260807 11:28:05.960556 3689 heartbeater.cc:464] Registering TS with master...
I20260807 11:28:05.961431 3689 heartbeater.cc:511] Master 127.16.197.62:33133 requested a full tablet report, sending...
I20260807 11:28:05.963780 3493 ts_manager.cc:194] Registered new tserver with Master: 2d40f11232d84fcd99be03bb8c4439b8 (127.16.197.1:34965)
I20260807 11:28:05.965994 3493 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.197.1:43021
W20260807 11:28:06.299225 3693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:06.299504 3693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:06.299681 3693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:06.307256 3693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:06.307368 3693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:06.307559 3693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:28:06.317152 3693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:44519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:33133
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:06.318504 3693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:06.320340 3693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:06.333927 3699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:06.333987 3703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:06.334012 3700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:06.334841 3693 server_base.cc:1034] running on GCE node
I20260807 11:28:06.335523 3693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:06.336526 3693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:06.337782 3693 hybrid_clock.cc:648] HybridClock initialized: now 1786102086337717 us; error 57 us; skew 500 ppm
I20260807 11:28:06.338127 3693 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 11:28:06.340520 3693 webserver.cc:460] Webserver started at http://127.16.197.2:45107/ using document root <none> and password file <none>
I20260807 11:28:06.341228 3693 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:06.341363 3693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:06.341674 3693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:06.344750 3693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-1/data/instance:
uuid: "cb028e2d6dcd4f1eb448b32a2e6e1f45"
format_stamp: "Formatted at 2026-08-07 11:28:06 on dist-test-slave-2w3w"
server_key: "1556003aabd680d4fd2988d9c4b87b27"
server_key_iv: "36dbd6f18821190e5a3a1ebf77695072"
server_key_version: "encryptionkey@0"
I20260807 11:28:06.345556 3693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance:
uuid: "cb028e2d6dcd4f1eb448b32a2e6e1f45"
format_stamp: "Formatted at 2026-08-07 11:28:06 on dist-test-slave-2w3w"
server_key: "1556003aabd680d4fd2988d9c4b87b27"
server_key_iv: "36dbd6f18821190e5a3a1ebf77695072"
server_key_version: "encryptionkey@0"
I20260807 11:28:06.350931 3693 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 11:28:06.354748 3709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:06.356235 3693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:28:06.356438 3693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "cb028e2d6dcd4f1eb448b32a2e6e1f45"
format_stamp: "Formatted at 2026-08-07 11:28:06 on dist-test-slave-2w3w"
server_key: "1556003aabd680d4fd2988d9c4b87b27"
server_key_iv: "36dbd6f18821190e5a3a1ebf77695072"
server_key_version: "encryptionkey@0"
I20260807 11:28:06.356674 3693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:06.382041 3693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:06.383059 3693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:06.383405 3693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:06.384456 3693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:28:06.386039 3693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:28:06.386171 3693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:06.386312 3693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:28:06.386401 3693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:06.420991 3693 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:46315
I20260807 11:28:06.421084 3829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:46315 every 8 connection(s)
I20260807 11:28:06.422328 3693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:28:06.423808 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 3693
I20260807 11:28:06.423987 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance
I20260807 11:28:06.428623 17172 external_mini_cluster.cc:1428] Setting key 3f7c2a1081fcaafed703a2f3ee92510d
I20260807 11:28:06.432056 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:0
--local_ip_for_outbound_sockets=127.16.197.3
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:33133
--builtin_ntp_servers=127.16.197.20:44519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 11:28:06.436623 3691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:06.425072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:48129 (local address 127.16.197.62:33133)
0807 11:28:06.425334 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:06.425343 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:06.426871 (+ 1528us) server_negotiation.cc:407] Connection header received
0807 11:28:06.428902 (+ 2031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:06.428909 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:06.428997 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:06.429128 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:06.430543 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:06.431650 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:06.433460 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:06.434198 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:06.434335 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:06.435705 (+ 1370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:06.435714 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:06.435722 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:06.435749 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:06.435791 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:06.435810 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:06.435908 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:06.436120 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 11:28:06.436232 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 11:28:06.437496 3832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:06.425532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33133 (local address 127.16.197.2:48129)
0807 11:28:06.426690 (+ 1158us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:06.426736 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:06.428593 (+ 1857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:06.429369 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:06.429382 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:06.429502 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:28:06.430255 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:06.430279 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:06.431805 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:06.431809 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:06.433295 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:06.433307 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:06.434414 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:06.434419 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:06.434617 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:06.435375 (+ 758us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 11:28:06.435378 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:28:06.435422 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 11:28:06.435500 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:28:06.435908 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:28:06.435916 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:28:06.435922 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 11:28:06.436078 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 11:28:06.436379 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":440,"threads_started":1}
I20260807 11:28:06.439024 3830 heartbeater.cc:347] Connected to a master server at 127.16.197.62:33133
I20260807 11:28:06.439334 3830 heartbeater.cc:464] Registering TS with master...
I20260807 11:28:06.440040 3830 heartbeater.cc:511] Master 127.16.197.62:33133 requested a full tablet report, sending...
I20260807 11:28:06.441458 3493 ts_manager.cc:194] Registered new tserver with Master: cb028e2d6dcd4f1eb448b32a2e6e1f45 (127.16.197.2:46315)
I20260807 11:28:06.442308 3493 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.197.2:48129
W20260807 11:28:06.769318 3833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:06.769585 3833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:06.769670 3833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:06.777076 3833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:06.777187 3833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:06.777298 3833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:28:06.786375 3833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:44519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:33133
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:06.787756 3833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:06.789539 3833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:06.803100 3842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:06.803081 3840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:06.803208 3839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:06.804076 3833 server_base.cc:1034] running on GCE node
I20260807 11:28:06.804790 3833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:06.805851 3833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:06.807056 3833 hybrid_clock.cc:648] HybridClock initialized: now 1786102086806982 us; error 56 us; skew 500 ppm
I20260807 11:28:06.807473 3833 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 11:28:06.810326 3833 webserver.cc:460] Webserver started at http://127.16.197.3:45363/ using document root <none> and password file <none>
I20260807 11:28:06.811342 3833 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:06.811497 3833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:06.811923 3833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:06.814877 3833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-2/data/instance:
uuid: "51790a484fd34612b008b30c529e8ffb"
format_stamp: "Formatted at 2026-08-07 11:28:06 on dist-test-slave-2w3w"
server_key: "f2929a42bff227ca855d3fca0a9b1732"
server_key_iv: "7bd5a53b6bdcc72b7f837ac54e5bd96e"
server_key_version: "encryptionkey@0"
I20260807 11:28:06.815985 3833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance:
uuid: "51790a484fd34612b008b30c529e8ffb"
format_stamp: "Formatted at 2026-08-07 11:28:06 on dist-test-slave-2w3w"
server_key: "f2929a42bff227ca855d3fca0a9b1732"
server_key_iv: "7bd5a53b6bdcc72b7f837ac54e5bd96e"
server_key_version: "encryptionkey@0"
I20260807 11:28:06.821213 3833 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 11:28:06.824582 3849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:06.825951 3833 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 11:28:06.826120 3833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "51790a484fd34612b008b30c529e8ffb"
format_stamp: "Formatted at 2026-08-07 11:28:06 on dist-test-slave-2w3w"
server_key: "f2929a42bff227ca855d3fca0a9b1732"
server_key_iv: "7bd5a53b6bdcc72b7f837ac54e5bd96e"
server_key_version: "encryptionkey@0"
I20260807 11:28:06.826344 3833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:06.845302 3833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:06.846119 3833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:06.846451 3833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:06.847548 3833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:28:06.848984 3833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:28:06.849135 3833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:06.849267 3833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:28:06.849362 3833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:06.884862 3833 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:40683
I20260807 11:28:06.884918 3966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:40683 every 8 connection(s)
I20260807 11:28:06.886291 3833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:28:06.891605 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 3833
I20260807 11:28:06.891834 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance
I20260807 11:28:06.892298 17172 external_mini_cluster.cc:1428] Setting key d8b8b06895d80de0af7715e020b13d18
I20260807 11:28:06.899451 3691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:06.889081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:57195 (local address 127.16.197.62:33133)
0807 11:28:06.889316 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:06.889322 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:06.890737 (+ 1415us) server_negotiation.cc:407] Connection header received
0807 11:28:06.892723 (+ 1986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:06.892731 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:06.892823 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:06.892911 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:06.894226 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:06.895152 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:06.896643 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:06.897108 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:06.897244 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:06.898550 (+ 1306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:06.898564 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:06.898567 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:06.898586 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:06.898617 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:06.898629 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:06.898728 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:06.899002 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 11:28:06.899103 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 11:28:06.900499 3969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:06.889668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33133 (local address 127.16.197.3:57195)
0807 11:28:06.890570 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:06.890604 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:06.892424 (+ 1820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:06.893157 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:06.893165 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:06.893278 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:28:06.894032 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:06.894057 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:06.895298 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:06.895304 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:06.896498 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:06.896510 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:06.897327 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:06.897334 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:06.897564 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:06.898206 (+ 642us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 11:28:06.898208 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:28:06.898265 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 11:28:06.898355 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:28:06.898727 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:28:06.898740 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:28:06.898750 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 11:28:06.898966 (+ 216us) client_negotiation.cc:240] Negotiation successful
0807 11:28:06.899384 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":297,"threads_started":1}
I20260807 11:28:06.901880 3967 heartbeater.cc:347] Connected to a master server at 127.16.197.62:33133
I20260807 11:28:06.902148 3967 heartbeater.cc:464] Registering TS with master...
I20260807 11:28:06.903173 3967 heartbeater.cc:511] Master 127.16.197.62:33133 requested a full tablet report, sending...
I20260807 11:28:06.904739 3493 ts_manager.cc:194] Registered new tserver with Master: 51790a484fd34612b008b30c529e8ffb (127.16.197.3:40683)
I20260807 11:28:06.905671 3493 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.197.3:57195
I20260807 11:28:06.907899 17172 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 11:28:06.918515 3691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:06.912209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36892 (local address 127.16.197.62:33133)
0807 11:28:06.912439 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:06.912448 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:06.912865 (+ 417us) server_negotiation.cc:407] Connection header received
0807 11:28:06.913078 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:06.913084 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:06.913207 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:06.913323 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:06.913894 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:06.914605 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:06.916070 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:06.916484 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:06.916630 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:06.917706 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:06.917750 (+ 44us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:06.917753 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:06.917772 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:06.917805 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:06.917818 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:06.917920 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:06.918070 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 11:28:06.918193 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 11:28:06.922565 3493 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36892:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 11:28:06.930557 3493 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:28:06.972600 3689 heartbeater.cc:503] Master 127.16.197.62:33133 was elected leader, sending a full tablet report...
I20260807 11:28:06.973649 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:06.943452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:46315 (local address 127.0.0.1:39812)
0807 11:28:06.958170 (+ 14718us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:06.958230 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:06.958499 (+ 269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:06.962607 (+ 4108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:06.962620 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:06.962668 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:06.964997 (+ 2329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:06.965017 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:06.967461 (+ 2444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:06.967468 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:06.970108 (+ 2640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:06.970123 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:06.972491 (+ 2368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:06.972498 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:06.972788 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:06.972843 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 11:28:06.972993 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 11:28:06.973214 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14347,"mutex_wait_us":1,"thread_start_us":15288,"threads_started":1}
I20260807 11:28:06.974570 3982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:06.943681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39812 (local address 127.16.197.2:46315)
0807 11:28:06.961373 (+ 17692us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:06.961386 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:06.961440 (+ 54us) server_negotiation.cc:407] Connection header received
0807 11:28:06.961583 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:06.961594 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:06.961924 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:06.962178 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:06.965656 (+ 3478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:06.967308 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:06.970370 (+ 3062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:06.971945 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:06.972086 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:06.972347 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:06.973998 (+ 1651us) server_negotiation.cc:299] Negotiation successful
0807 11:28:06.974187 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17472,"thread_start_us":17330,"threads_started":1}
I20260807 11:28:06.976018 3984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:06.946282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:34965 (local address 127.0.0.1:50566)
0807 11:28:06.967300 (+ 21018us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:06.967319 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:06.967486 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:06.968471 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:06.968476 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:06.968502 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:06.968957 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:06.968972 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:06.970905 (+ 1933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:06.970911 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:06.973065 (+ 2154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:06.973077 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:06.975207 (+ 2130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:06.975214 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:06.975396 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:06.975438 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 11:28:06.975528 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 11:28:06.975665 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20891,"mutex_wait_us":41,"thread_start_us":20751,"threads_started":1}
I20260807 11:28:06.976122 3985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:06.947072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50566 (local address 127.16.197.1:34965)
0807 11:28:06.966815 (+ 19743us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:06.966827 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:06.967658 (+ 831us) server_negotiation.cc:407] Connection header received
0807 11:28:06.967769 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:06.967777 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:06.968016 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:06.968218 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:06.969175 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:06.970719 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:06.973294 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:06.975000 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:06.975142 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:06.975408 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:06.975615 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 11:28:06.975795 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19535,"thread_start_us":19416,"threads_started":1}
I20260807 11:28:06.978296 3981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:06.943603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:40683 (local address 127.0.0.1:50434)
0807 11:28:06.965660 (+ 22057us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:06.965682 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:06.965810 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:06.968056 (+ 2246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:06.968061 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:06.968089 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:06.968433 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:06.968442 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:06.970369 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:06.970377 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:06.974297 (+ 3920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:06.974310 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:06.977522 (+ 3212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:06.977531 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:06.977715 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:06.977748 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 11:28:06.977841 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 11:28:06.977985 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21911,"mutex_wait_us":1,"thread_start_us":21763,"threads_started":1}
I20260807 11:28:06.979176 3983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:06.944017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50434 (local address 127.16.197.3:40683)
0807 11:28:06.966414 (+ 22397us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:06.966433 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:06.966469 (+ 36us) server_negotiation.cc:407] Connection header received
0807 11:28:06.966563 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:06.966571 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:06.967512 (+ 941us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:06.967721 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:06.968628 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:06.970188 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:06.974584 (+ 4396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:06.976774 (+ 2190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:06.976944 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:06.977209 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:06.978574 (+ 1365us) server_negotiation.cc:299] Negotiation successful
0807 11:28:06.978752 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22213,"thread_start_us":22042,"threads_started":1}
I20260807 11:28:06.979054 3761 tablet_service.cc:1467] Processing CreateTablet for tablet a97dab989c5b42ff9d11104702b4dc25 (DEFAULT_TABLE table=test-table [id=79a06ad98ae2429a997ac4d45eebe27c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:28:06.979674 3620 tablet_service.cc:1467] Processing CreateTablet for tablet a97dab989c5b42ff9d11104702b4dc25 (DEFAULT_TABLE table=test-table [id=79a06ad98ae2429a997ac4d45eebe27c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:28:06.980623 3761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a97dab989c5b42ff9d11104702b4dc25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:06.981123 3620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a97dab989c5b42ff9d11104702b4dc25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:06.982806 3899 tablet_service.cc:1467] Processing CreateTablet for tablet a97dab989c5b42ff9d11104702b4dc25 (DEFAULT_TABLE table=test-table [id=79a06ad98ae2429a997ac4d45eebe27c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:28:06.984812 3899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a97dab989c5b42ff9d11104702b4dc25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:06.992868 3986 tablet_bootstrap.cc:492] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45: Bootstrap starting.
I20260807 11:28:06.996124 3986 tablet_bootstrap.cc:654] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:06.997005 3987 tablet_bootstrap.cc:492] T a97dab989c5b42ff9d11104702b4dc25 P 2d40f11232d84fcd99be03bb8c4439b8: Bootstrap starting.
I20260807 11:28:06.997452 3986 log.cc:824] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:07.000852 3986 tablet_bootstrap.cc:492] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45: No bootstrap required, opened a new log
I20260807 11:28:07.000895 3989 tablet_bootstrap.cc:492] T a97dab989c5b42ff9d11104702b4dc25 P 51790a484fd34612b008b30c529e8ffb: Bootstrap starting.
I20260807 11:28:07.001178 3986 ts_tablet_manager.cc:1397] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260807 11:28:07.001906 3987 tablet_bootstrap.cc:654] T a97dab989c5b42ff9d11104702b4dc25 P 2d40f11232d84fcd99be03bb8c4439b8: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:07.003211 3987 log.cc:824] T a97dab989c5b42ff9d11104702b4dc25 P 2d40f11232d84fcd99be03bb8c4439b8: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:07.004288 3989 tablet_bootstrap.cc:654] T a97dab989c5b42ff9d11104702b4dc25 P 51790a484fd34612b008b30c529e8ffb: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:07.005448 3989 log.cc:824] T a97dab989c5b42ff9d11104702b4dc25 P 51790a484fd34612b008b30c529e8ffb: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:07.006466 3987 tablet_bootstrap.cc:492] T a97dab989c5b42ff9d11104702b4dc25 P 2d40f11232d84fcd99be03bb8c4439b8: No bootstrap required, opened a new log
I20260807 11:28:07.006757 3987 ts_tablet_manager.cc:1397] T a97dab989c5b42ff9d11104702b4dc25 P 2d40f11232d84fcd99be03bb8c4439b8: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260807 11:28:07.008193 3989 tablet_bootstrap.cc:492] T a97dab989c5b42ff9d11104702b4dc25 P 51790a484fd34612b008b30c529e8ffb: No bootstrap required, opened a new log
I20260807 11:28:07.008478 3989 ts_tablet_manager.cc:1397] T a97dab989c5b42ff9d11104702b4dc25 P 51790a484fd34612b008b30c529e8ffb: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 11:28:07.011880 3986 raft_consensus.cc:348] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb028e2d6dcd4f1eb448b32a2e6e1f45" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46315 } } peers { permanent_uuid: "51790a484fd34612b008b30c529e8ffb" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40683 } } peers { permanent_uuid: "2d40f11232d84fcd99be03bb8c4439b8" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 34965 } }
I20260807 11:28:07.012356 3986 raft_consensus.cc:374] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:07.012502 3986 raft_consensus.cc:729] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb028e2d6dcd4f1eb448b32a2e6e1f45, State: Initialized, Role: FOLLOWER
I20260807 11:28:07.013170 3986 consensus_queue.cc:260] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb028e2d6dcd4f1eb448b32a2e6e1f45" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46315 } } peers { permanent_uuid: "51790a484fd34612b008b30c529e8ffb" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40683 } } peers { permanent_uuid: "2d40f11232d84fcd99be03bb8c4439b8" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 34965 } }
I20260807 11:28:07.014354 3830 heartbeater.cc:503] Master 127.16.197.62:33133 was elected leader, sending a full tablet report...
I20260807 11:28:07.015126 3986 ts_tablet_manager.cc:1428] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260807 11:28:07.017978 3989 raft_consensus.cc:348] T a97dab989c5b42ff9d11104702b4dc25 P 51790a484fd34612b008b30c529e8ffb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb028e2d6dcd4f1eb448b32a2e6e1f45" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46315 } } peers { permanent_uuid: "51790a484fd34612b008b30c529e8ffb" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40683 } } peers { permanent_uuid: "2d40f11232d84fcd99be03bb8c4439b8" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 34965 } }
I20260807 11:28:07.018004 3987 raft_consensus.cc:348] T a97dab989c5b42ff9d11104702b4dc25 P 2d40f11232d84fcd99be03bb8c4439b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb028e2d6dcd4f1eb448b32a2e6e1f45" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46315 } } peers { permanent_uuid: "51790a484fd34612b008b30c529e8ffb" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40683 } } peers { permanent_uuid: "2d40f11232d84fcd99be03bb8c4439b8" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 34965 } }
I20260807 11:28:07.018374 3989 raft_consensus.cc:374] T a97dab989c5b42ff9d11104702b4dc25 P 51790a484fd34612b008b30c529e8ffb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:07.018476 3989 raft_consensus.cc:729] T a97dab989c5b42ff9d11104702b4dc25 P 51790a484fd34612b008b30c529e8ffb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51790a484fd34612b008b30c529e8ffb, State: Initialized, Role: FOLLOWER
I20260807 11:28:07.018471 3987 raft_consensus.cc:374] T a97dab989c5b42ff9d11104702b4dc25 P 2d40f11232d84fcd99be03bb8c4439b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:07.018584 3987 raft_consensus.cc:729] T a97dab989c5b42ff9d11104702b4dc25 P 2d40f11232d84fcd99be03bb8c4439b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d40f11232d84fcd99be03bb8c4439b8, State: Initialized, Role: FOLLOWER
I20260807 11:28:07.019150 3989 consensus_queue.cc:260] T a97dab989c5b42ff9d11104702b4dc25 P 51790a484fd34612b008b30c529e8ffb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb028e2d6dcd4f1eb448b32a2e6e1f45" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46315 } } peers { permanent_uuid: "51790a484fd34612b008b30c529e8ffb" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40683 } } peers { permanent_uuid: "2d40f11232d84fcd99be03bb8c4439b8" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 34965 } }
I20260807 11:28:07.019297 3987 consensus_queue.cc:260] T a97dab989c5b42ff9d11104702b4dc25 P 2d40f11232d84fcd99be03bb8c4439b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb028e2d6dcd4f1eb448b32a2e6e1f45" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46315 } } peers { permanent_uuid: "51790a484fd34612b008b30c529e8ffb" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40683 } } peers { permanent_uuid: "2d40f11232d84fcd99be03bb8c4439b8" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 34965 } }
I20260807 11:28:07.021204 3989 ts_tablet_manager.cc:1428] T a97dab989c5b42ff9d11104702b4dc25 P 51790a484fd34612b008b30c529e8ffb: Time spent starting tablet: real 0.013s user 0.008s sys 0.005s
I20260807 11:28:07.021726 3987 ts_tablet_manager.cc:1428] T a97dab989c5b42ff9d11104702b4dc25 P 2d40f11232d84fcd99be03bb8c4439b8: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260807 11:28:07.022434 3967 heartbeater.cc:503] Master 127.16.197.62:33133 was elected leader, sending a full tablet report...
I20260807 11:28:07.036453 3992 raft_consensus.cc:482] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:28:07.036826 3992 raft_consensus.cc:504] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb028e2d6dcd4f1eb448b32a2e6e1f45" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46315 } } peers { permanent_uuid: "51790a484fd34612b008b30c529e8ffb" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40683 } } peers { permanent_uuid: "2d40f11232d84fcd99be03bb8c4439b8" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 34965 } }
I20260807 11:28:07.038515 3992 leader_election.cc:288] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51790a484fd34612b008b30c529e8ffb (127.16.197.3:40683), 2d40f11232d84fcd99be03bb8c4439b8 (127.16.197.1:34965)
I20260807 11:28:07.046300 3985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:07.039227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:36889 (local address 127.16.197.1:34965)
0807 11:28:07.039442 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:07.039448 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:07.039886 (+ 438us) server_negotiation.cc:407] Connection header received
0807 11:28:07.040193 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:07.040200 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:07.040296 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:07.040407 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:07.041126 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:07.041895 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:07.044073 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:07.045312 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:07.045506 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:07.045596 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:07.045962 (+ 366us) server_negotiation.cc:299] Negotiation successful
0807 11:28:07.046091 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 11:28:07.046327 3996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:07.038760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:40683 (local address 127.16.197.2:45489)
0807 11:28:07.039954 (+ 1194us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:07.039975 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:07.040113 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:07.040601 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:07.040605 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:07.040642 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:07.041177 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:07.041192 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:07.042195 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:07.042200 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:07.043806 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:07.043819 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:07.045377 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:07.045385 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:07.045712 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:07.045749 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 11:28:07.045847 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 11:28:07.045986 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1070,"mutex_wait_us":180,"thread_start_us":336,"threads_started":1}
I20260807 11:28:07.046819 3983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:07.038788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:45489 (local address 127.16.197.3:40683)
0807 11:28:07.039052 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:07.039062 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:07.040052 (+ 990us) server_negotiation.cc:407] Connection header received
0807 11:28:07.040252 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:07.040257 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:07.040331 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:07.040416 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:07.041337 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:07.042043 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:07.044009 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:07.045124 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:07.045313 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:07.045390 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:07.045936 (+ 546us) server_negotiation.cc:299] Negotiation successful
0807 11:28:07.046186 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 11:28:07.047439 3997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:07.038749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:34965 (local address 127.16.197.2:36889)
0807 11:28:07.039750 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:07.039770 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:07.039965 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:07.040437 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:07.040441 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:07.040468 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:07.040930 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:07.040944 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:07.042044 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:07.042048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:07.043860 (+ 1812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:07.043875 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:07.045522 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:07.045527 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:07.045762 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:07.045792 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 11:28:07.046842 (+ 1050us) client_negotiation.cc:240] Negotiation successful
0807 11:28:07.047023 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":825,"mutex_wait_us":123,"spinlock_wait_cycles":2432,"thread_start_us":843,"threads_started":1}
I20260807 11:28:07.048810 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a97dab989c5b42ff9d11104702b4dc25" candidate_uuid: "cb028e2d6dcd4f1eb448b32a2e6e1f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51790a484fd34612b008b30c529e8ffb" is_pre_election: true
I20260807 11:28:07.049237 3641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a97dab989c5b42ff9d11104702b4dc25" candidate_uuid: "cb028e2d6dcd4f1eb448b32a2e6e1f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d40f11232d84fcd99be03bb8c4439b8" is_pre_election: true
I20260807 11:28:07.049494 3920 raft_consensus.cc:2454] T a97dab989c5b42ff9d11104702b4dc25 P 51790a484fd34612b008b30c529e8ffb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb028e2d6dcd4f1eb448b32a2e6e1f45 in term 0.
I20260807 11:28:07.049829 3641 raft_consensus.cc:2454] T a97dab989c5b42ff9d11104702b4dc25 P 2d40f11232d84fcd99be03bb8c4439b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb028e2d6dcd4f1eb448b32a2e6e1f45 in term 0.
I20260807 11:28:07.050391 3711 leader_election.cc:302] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51790a484fd34612b008b30c529e8ffb, cb028e2d6dcd4f1eb448b32a2e6e1f45; no voters:
I20260807 11:28:07.050791 3992 raft_consensus.cc:2781] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:28:07.050944 3992 raft_consensus.cc:482] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:28:07.051083 3992 raft_consensus.cc:3037] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:07.053401 3992 raft_consensus.cc:504] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb028e2d6dcd4f1eb448b32a2e6e1f45" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46315 } } peers { permanent_uuid: "51790a484fd34612b008b30c529e8ffb" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40683 } } peers { permanent_uuid: "2d40f11232d84fcd99be03bb8c4439b8" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 34965 } }
I20260807 11:28:07.054028 3992 leader_election.cc:288] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45 [CANDIDATE]: Term 1 election: Requested vote from peers 51790a484fd34612b008b30c529e8ffb (127.16.197.3:40683), 2d40f11232d84fcd99be03bb8c4439b8 (127.16.197.1:34965)
I20260807 11:28:07.054476 3920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a97dab989c5b42ff9d11104702b4dc25" candidate_uuid: "cb028e2d6dcd4f1eb448b32a2e6e1f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51790a484fd34612b008b30c529e8ffb"
I20260807 11:28:07.054587 3641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a97dab989c5b42ff9d11104702b4dc25" candidate_uuid: "cb028e2d6dcd4f1eb448b32a2e6e1f45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d40f11232d84fcd99be03bb8c4439b8"
I20260807 11:28:07.054693 3920 raft_consensus.cc:3037] T a97dab989c5b42ff9d11104702b4dc25 P 51790a484fd34612b008b30c529e8ffb [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:07.054811 3641 raft_consensus.cc:3037] T a97dab989c5b42ff9d11104702b4dc25 P 2d40f11232d84fcd99be03bb8c4439b8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:07.058135 3920 raft_consensus.cc:2454] T a97dab989c5b42ff9d11104702b4dc25 P 51790a484fd34612b008b30c529e8ffb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb028e2d6dcd4f1eb448b32a2e6e1f45 in term 1.
I20260807 11:28:07.058135 3641 raft_consensus.cc:2454] T a97dab989c5b42ff9d11104702b4dc25 P 2d40f11232d84fcd99be03bb8c4439b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb028e2d6dcd4f1eb448b32a2e6e1f45 in term 1.
I20260807 11:28:07.058609 3710 leader_election.cc:302] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2d40f11232d84fcd99be03bb8c4439b8, 51790a484fd34612b008b30c529e8ffb, cb028e2d6dcd4f1eb448b32a2e6e1f45; no voters:
I20260807 11:28:07.058980 3992 raft_consensus.cc:2781] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:07.059425 3992 raft_consensus.cc:686] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45 [term 1 LEADER]: Becoming Leader. State: Replica: cb028e2d6dcd4f1eb448b32a2e6e1f45, State: Running, Role: LEADER
I20260807 11:28:07.060025 3992 consensus_queue.cc:237] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb028e2d6dcd4f1eb448b32a2e6e1f45" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46315 } } peers { permanent_uuid: "51790a484fd34612b008b30c529e8ffb" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40683 } } peers { permanent_uuid: "2d40f11232d84fcd99be03bb8c4439b8" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 34965 } }
I20260807 11:28:07.065181 3493 catalog_manager.cc:5495] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb028e2d6dcd4f1eb448b32a2e6e1f45 (127.16.197.2). New cstate: current_term: 1 leader_uuid: "cb028e2d6dcd4f1eb448b32a2e6e1f45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb028e2d6dcd4f1eb448b32a2e6e1f45" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 46315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51790a484fd34612b008b30c529e8ffb" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d40f11232d84fcd99be03bb8c4439b8" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 34965 } health_report { overall_health: UNKNOWN } } }
I20260807 11:28:07.097280 3982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:07.091703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39818 (local address 127.16.197.2:46315)
0807 11:28:07.091997 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:07.092005 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:07.092041 (+ 36us) server_negotiation.cc:407] Connection header received
0807 11:28:07.092140 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:07.092144 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:07.092251 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:07.092393 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:28:07.092728 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:07.093498 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:07.095158 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:07.095595 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:07.095706 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:07.096138 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:28:07.096659 (+ 521us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:28:07.096788 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:07.096852 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 11:28:07.096921 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260807 11:28:07.106124 3920 raft_consensus.cc:1261] T a97dab989c5b42ff9d11104702b4dc25 P 51790a484fd34612b008b30c529e8ffb [term 1 FOLLOWER]: Refusing update from remote peer cb028e2d6dcd4f1eb448b32a2e6e1f45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:28:07.106117 3641 raft_consensus.cc:1261] T a97dab989c5b42ff9d11104702b4dc25 P 2d40f11232d84fcd99be03bb8c4439b8 [term 1 FOLLOWER]: Refusing update from remote peer cb028e2d6dcd4f1eb448b32a2e6e1f45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:28:07.107214 3998 consensus_queue.cc:1035] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51790a484fd34612b008b30c529e8ffb" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 40683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:28:07.107589 3992 consensus_queue.cc:1035] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d40f11232d84fcd99be03bb8c4439b8" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 34965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:28:07.118975 4001 mvcc.cc:205] Tried to move back new op lower bound from 7315874148775444480 to 7315874148602863616. Current Snapshot: MvccSnapshot[applied={T|T < 7315874148775444480}]
I20260807 11:28:07.120114 4002 mvcc.cc:205] Tried to move back new op lower bound from 7315874148775444480 to 7315874148602863616. Current Snapshot: MvccSnapshot[applied={T|T < 7315874148775444480}]
I20260807 11:28:07.129920 4003 mvcc.cc:205] Tried to move back new op lower bound from 7315874148775444480 to 7315874148602863616. Current Snapshot: MvccSnapshot[applied={T|T < 7315874148775444480 or (T in {7315874148775444480})}]
I20260807 11:28:07.134478 3492 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:28:07.134840 3492 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:28:07.139101 3492 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 405440a089db4eb09b8d82350eb2e9a1: Sending DeleteTablet for 3 replicas of tablet a97dab989c5b42ff9d11104702b4dc25
I20260807 11:28:07.140391 3761 tablet_service.cc:1514] Processing DeleteTablet for tablet a97dab989c5b42ff9d11104702b4dc25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:28:07 UTC) from {username='slave'} at 127.0.0.1:39812
I20260807 11:28:07.140467 3899 tablet_service.cc:1514] Processing DeleteTablet for tablet a97dab989c5b42ff9d11104702b4dc25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:28:07 UTC) from {username='slave'} at 127.0.0.1:50434
I20260807 11:28:07.140491 3620 tablet_service.cc:1514] Processing DeleteTablet for tablet a97dab989c5b42ff9d11104702b4dc25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:28:07 UTC) from {username='slave'} at 127.0.0.1:50566
I20260807 11:28:07.143754 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 3556
I20260807 11:28:07.160652 4011 tablet_replica.cc:330] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45: stopping tablet replica
I20260807 11:28:07.160041 4012 tablet_replica.cc:330] T a97dab989c5b42ff9d11104702b4dc25 P 51790a484fd34612b008b30c529e8ffb: stopping tablet replica
I20260807 11:28:07.161725 4012 raft_consensus.cc:2229] T a97dab989c5b42ff9d11104702b4dc25 P 51790a484fd34612b008b30c529e8ffb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:28:07.162515 4011 raft_consensus.cc:2229] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45 [term 1 LEADER]: Raft consensus shutting down.
I20260807 11:28:07.162653 4013 tablet_replica.cc:330] T a97dab989c5b42ff9d11104702b4dc25 P 2d40f11232d84fcd99be03bb8c4439b8: stopping tablet replica
I20260807 11:28:07.163066 4013 raft_consensus.cc:2229] T a97dab989c5b42ff9d11104702b4dc25 P 2d40f11232d84fcd99be03bb8c4439b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:28:07.163323 4011 raft_consensus.cc:2258] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:28:07.163587 4013 raft_consensus.cc:2258] T a97dab989c5b42ff9d11104702b4dc25 P 2d40f11232d84fcd99be03bb8c4439b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:28:07.164091 4012 raft_consensus.cc:2258] T a97dab989c5b42ff9d11104702b4dc25 P 51790a484fd34612b008b30c529e8ffb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:28:07.166162 3985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:07.146720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50580 (local address 127.16.197.1:34965)
0807 11:28:07.146942 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:07.146951 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:07.150194 (+ 3243us) server_negotiation.cc:407] Connection header received
0807 11:28:07.151176 (+ 982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:07.151186 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:07.151360 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:07.151461 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:07.152883 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:07.154549 (+ 1666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:07.159935 (+ 5386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:07.160688 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:07.160807 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:07.164415 (+ 3608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:07.164426 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:07.164444 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:07.165226 (+ 782us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:07.165292 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:07.165319 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:07.165442 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:07.165669 (+ 227us) server_negotiation.cc:299] Negotiation successful
0807 11:28:07.165795 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 11:28:07.166280 4011 ts_tablet_manager.cc:1905] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:28:07.166396 4012 ts_tablet_manager.cc:1905] T a97dab989c5b42ff9d11104702b4dc25 P 51790a484fd34612b008b30c529e8ffb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:28:07.167696 3684 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:07.178179 4011 ts_tablet_manager.cc:1918] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:28:07.178435 4011 log.cc:1196] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-1/wal/wals/a97dab989c5b42ff9d11104702b4dc25
I20260807 11:28:07.178812 4012 ts_tablet_manager.cc:1918] T a97dab989c5b42ff9d11104702b4dc25 P 51790a484fd34612b008b30c529e8ffb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:28:07.179092 4012 log.cc:1196] T a97dab989c5b42ff9d11104702b4dc25 P 51790a484fd34612b008b30c529e8ffb: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786101765888975-17172-0/minicluster-data/ts-2/wal/wals/a97dab989c5b42ff9d11104702b4dc25
I20260807 11:28:07.179709 4012 ts_tablet_manager.cc:1939] T a97dab989c5b42ff9d11104702b4dc25 P 51790a484fd34612b008b30c529e8ffb: Deleting consensus metadata
I20260807 11:28:07.180079 4011 ts_tablet_manager.cc:1939] T a97dab989c5b42ff9d11104702b4dc25 P cb028e2d6dcd4f1eb448b32a2e6e1f45: Deleting consensus metadata
I20260807 11:28:07.181905 3478 catalog_manager.cc:4841] TS 51790a484fd34612b008b30c529e8ffb (127.16.197.3:40683): tablet a97dab989c5b42ff9d11104702b4dc25 (table test-table [id=79a06ad98ae2429a997ac4d45eebe27c]) successfully deleted
I20260807 11:28:07.182027 3479 catalog_manager.cc:4841] TS cb028e2d6dcd4f1eb448b32a2e6e1f45 (127.16.197.2:46315): tablet a97dab989c5b42ff9d11104702b4dc25 (table test-table [id=79a06ad98ae2429a997ac4d45eebe27c]) successfully deleted
I20260807 11:28:07.168988 4013 ts_tablet_manager.cc:1905] T a97dab989c5b42ff9d11104702b4dc25 P 2d40f11232d84fcd99be03bb8c4439b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:28:07.268630 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 3556
W20260807 11:28:07.295568 3477 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.197.1:34965 (error 108)
I20260807 11:28:07.295797 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 3693
I20260807 11:28:07.297915 3981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:07.297384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:34965 (local address 127.0.0.1:50590)
0807 11:28:07.297581 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:07.297716 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:34965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260807 11:28:07.298254 3477 catalog_manager.cc:4584] TS 2d40f11232d84fcd99be03bb8c4439b8 (127.16.197.1:34965): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a97dab989c5b42ff9d11104702b4dc25: Network error: Client connection negotiation failed: client connection to 127.16.197.1:34965: connect: Connection refused (error 111)
I20260807 11:28:07.305023 3982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:07.297468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39834 (local address 127.16.197.2:46315)
0807 11:28:07.297645 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:07.297652 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:07.297672 (+ 20us) server_negotiation.cc:407] Connection header received
0807 11:28:07.297741 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:07.297746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:07.297864 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:07.297986 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:07.298488 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:07.299629 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:07.301707 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:07.302146 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:07.302243 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:07.303485 (+ 1242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:07.303493 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:07.303511 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:07.304159 (+ 648us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:07.304212 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:07.304232 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:07.304320 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:07.304480 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 11:28:07.304616 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 11:28:07.305830 3825 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:07.352101 3981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:07.351633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:34965 (local address 127.0.0.1:50602)
0807 11:28:07.351842 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:07.351932 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:34965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 11:28:07.353144 3981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:07.352785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:34965 (local address 127.0.0.1:50614)
0807 11:28:07.352941 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:07.353019 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:34965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 11:28:07.413508 3981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:07.412989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:34965 (local address 127.0.0.1:50624)
0807 11:28:07.413224 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:07.413309 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:34965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260807 11:28:07.414587 3981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:07.414231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:34965 (local address 127.0.0.1:50640)
0807 11:28:07.414398 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:07.414472 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:34965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 11:28:07.420042 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 3693
I20260807 11:28:07.440625 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 3833
I20260807 11:28:07.447278 3983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:07.441417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50446 (local address 127.16.197.3:40683)
0807 11:28:07.441582 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:07.441586 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:07.441605 (+ 19us) server_negotiation.cc:407] Connection header received
0807 11:28:07.441773 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:07.441776 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:07.441853 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:07.441945 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:07.442329 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:07.443156 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:07.444431 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:07.444807 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:07.444897 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:07.445932 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:07.445941 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:07.445955 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:07.446520 (+ 565us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:07.446567 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:07.446586 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:07.446677 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:07.446785 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 11:28:07.446865 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 11:28:07.447889 3962 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:07.495321 3981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:07.494812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:34965 (local address 127.0.0.1:50648)
0807 11:28:07.495061 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:07.495156 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:34965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 11:28:07.496434 3981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:07.496038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:34965 (local address 127.0.0.1:50664)
0807 11:28:07.496236 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:07.496304 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:34965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260807 11:28:07.538167 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 3833
I20260807 11:28:07.556500 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 3461
I20260807 11:28:07.557647 3524 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:07.670939 3981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:07.670365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:34965 (local address 127.0.0.1:50668)
0807 11:28:07.670601 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:07.670728 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:34965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 11:28:07.671069 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 3461
2026-08-07T11:28:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2843 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:28:07 dist-test-slave-2w3w krb5kdc[4022](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:28:07 dist-test-slave-2w3w krb5kdc[4022](info): set up 2 sockets
Aug 07 11:28:07 dist-test-slave-2w3w krb5kdc[4022](info): commencing operation
krb5kdc: starting...
W20260807 11:28:08.826064 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.002s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:28:09 dist-test-slave-2w3w krb5kdc[4022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:28:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:28:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:28:09.122815 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:45631
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:45523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:45631
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:09.463182 4035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:09.463541 4035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:09.463702 4035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:09.471236 4035 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:28:09.471382 4035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:09.471495 4035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:09.471596 4035 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:28:09.471678 4035 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:28:09.481866 4035 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:45523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:45631
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:45631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:09.483316 4035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:09.484638 4035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:09.496884 4041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:09.496807 4044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:09.496855 4042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:09.497645 4035 server_base.cc:1034] running on GCE node
I20260807 11:28:09.498437 4035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:09.499995 4035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:09.501224 4035 hybrid_clock.cc:648] HybridClock initialized: now 1786102089501180 us; error 55 us; skew 500 ppm
I20260807 11:28:09.501557 4035 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:09 dist-test-slave-2w3w krb5kdc[4022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102089, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:09.504839 4035 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:28:09.506358 4035 webserver.cc:460] Webserver started at http://127.16.197.62:43267/ using document root <none> and password file <none>
I20260807 11:28:09.507117 4035 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:09.507261 4035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:09.507565 4035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:09.510610 4035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "5dfddbdb0c9d45b5ac0f176a3bc896f3"
format_stamp: "Formatted at 2026-08-07 11:28:09 on dist-test-slave-2w3w"
server_key: "b6caa874502bf2ca936d7f231292181b"
server_key_iv: "a8955e0fb818a2d0e02ef21cf14be89b"
server_key_version: "encryptionkey@0"
I20260807 11:28:09.511389 4035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "5dfddbdb0c9d45b5ac0f176a3bc896f3"
format_stamp: "Formatted at 2026-08-07 11:28:09 on dist-test-slave-2w3w"
server_key: "b6caa874502bf2ca936d7f231292181b"
server_key_iv: "a8955e0fb818a2d0e02ef21cf14be89b"
server_key_version: "encryptionkey@0"
I20260807 11:28:09.516590 4035 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 11:28:09.519942 4052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:09.521230 4035 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 11:28:09.521413 4035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "5dfddbdb0c9d45b5ac0f176a3bc896f3"
format_stamp: "Formatted at 2026-08-07 11:28:09 on dist-test-slave-2w3w"
server_key: "b6caa874502bf2ca936d7f231292181b"
server_key_iv: "a8955e0fb818a2d0e02ef21cf14be89b"
server_key_version: "encryptionkey@0"
I20260807 11:28:09.521615 4035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:09.538589 4035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:09.542897 4035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:09.543231 4035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:09.562209 4035 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:45631
I20260807 11:28:09.562263 4106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:45631 every 8 connection(s)
I20260807 11:28:09.563642 4035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:28:09.567443 4107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:09.570860 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 4035
I20260807 11:28:09.571103 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:28:09.571691 17172 external_mini_cluster.cc:1428] Setting key 9ce0825e7a01d8e0b947550938b83231
I20260807 11:28:09.580350 4107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dfddbdb0c9d45b5ac0f176a3bc896f3: Bootstrap starting.
I20260807 11:28:09.584249 4107 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5dfddbdb0c9d45b5ac0f176a3bc896f3: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:09.585588 4107 log.cc:824] T 00000000000000000000000000000000 P 5dfddbdb0c9d45b5ac0f176a3bc896f3: Log is configured to *not* fsync() on all Append() calls
Aug 07 11:28:09 dist-test-slave-2w3w krb5kdc[4022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:28:09.588676 4107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dfddbdb0c9d45b5ac0f176a3bc896f3: No bootstrap required, opened a new log
I20260807 11:28:09.596751 4111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:09.573572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56482 (local address 127.16.197.62:45631)
0807 11:28:09.574333 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:09.574345 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:09.574387 (+ 42us) server_negotiation.cc:407] Connection header received
0807 11:28:09.576115 (+ 1728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:09.576137 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:09.576533 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:09.576855 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:09.578759 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:09.580088 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:09.581953 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:09.582594 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:09.582712 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:09.588119 (+ 5407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:09.588127 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:09.588142 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:09.588177 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:09.592167 (+ 3990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:09.592855 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:09.592865 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:09.592874 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:09.593086 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:09.593436 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:09.593441 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:09.593444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:09.594178 (+ 734us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:09.594502 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:09.594808 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 11:28:09.595296 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":362,"threads_started":1}
I20260807 11:28:09.599170 4107 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5dfddbdb0c9d45b5ac0f176a3bc896f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dfddbdb0c9d45b5ac0f176a3bc896f3" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45631 } }
I20260807 11:28:09.599575 4107 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5dfddbdb0c9d45b5ac0f176a3bc896f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:09.599700 4107 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5dfddbdb0c9d45b5ac0f176a3bc896f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dfddbdb0c9d45b5ac0f176a3bc896f3, State: Initialized, Role: FOLLOWER
I20260807 11:28:09.600298 4107 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5dfddbdb0c9d45b5ac0f176a3bc896f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dfddbdb0c9d45b5ac0f176a3bc896f3" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45631 } }
I20260807 11:28:09.600608 4107 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5dfddbdb0c9d45b5ac0f176a3bc896f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:28:09.600772 4107 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5dfddbdb0c9d45b5ac0f176a3bc896f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:28:09.600976 4107 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5dfddbdb0c9d45b5ac0f176a3bc896f3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:09.603636 4107 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5dfddbdb0c9d45b5ac0f176a3bc896f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dfddbdb0c9d45b5ac0f176a3bc896f3" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45631 } }
I20260807 11:28:09.604174 4107 leader_election.cc:302] T 00000000000000000000000000000000 P 5dfddbdb0c9d45b5ac0f176a3bc896f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5dfddbdb0c9d45b5ac0f176a3bc896f3; no voters:
I20260807 11:28:09.604773 4107 leader_election.cc:288] T 00000000000000000000000000000000 P 5dfddbdb0c9d45b5ac0f176a3bc896f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:28:09.604924 4114 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5dfddbdb0c9d45b5ac0f176a3bc896f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:09.605640 4114 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5dfddbdb0c9d45b5ac0f176a3bc896f3 [term 1 LEADER]: Becoming Leader. State: Replica: 5dfddbdb0c9d45b5ac0f176a3bc896f3, State: Running, Role: LEADER
I20260807 11:28:09.606197 4114 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5dfddbdb0c9d45b5ac0f176a3bc896f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dfddbdb0c9d45b5ac0f176a3bc896f3" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45631 } }
I20260807 11:28:09.606895 4107 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5dfddbdb0c9d45b5ac0f176a3bc896f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:28:09.609246 4115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dfddbdb0c9d45b5ac0f176a3bc896f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5dfddbdb0c9d45b5ac0f176a3bc896f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dfddbdb0c9d45b5ac0f176a3bc896f3" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45631 } } }
I20260807 11:28:09.609671 4115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dfddbdb0c9d45b5ac0f176a3bc896f3 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:09.609198 4116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dfddbdb0c9d45b5ac0f176a3bc896f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5dfddbdb0c9d45b5ac0f176a3bc896f3. Latest consensus state: current_term: 1 leader_uuid: "5dfddbdb0c9d45b5ac0f176a3bc896f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dfddbdb0c9d45b5ac0f176a3bc896f3" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 45631 } } }
I20260807 11:28:09.610200 4116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dfddbdb0c9d45b5ac0f176a3bc896f3 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:09.611575 4122 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:28:09.616901 4122 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:28:09.627151 4122 catalog_manager.cc:1347] Generated new cluster ID: 1bcd8507fc5e44198bb496ad6c19ca9f
I20260807 11:28:09.627274 4122 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:28:09.651774 4122 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:28:09.653115 4122 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:28:09.662751 4122 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5dfddbdb0c9d45b5ac0f176a3bc896f3: Generated new TSK 0
I20260807 11:28:09.663931 4122 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.197.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
I20260807 11:28:09.743878 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:0
--local_ip_for_outbound_sockets=127.16.197.1
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:45631
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:45523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:10.092188 4136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:10.092465 4136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:10.092643 4136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:10.100133 4136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:10.100250 4136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:10.100456 4136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:28:10.110646 4136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:45523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:45631
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:10.112013 4136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:10.113430 4136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:10.127844 4142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:10.127925 4145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:10.127916 4143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:10.129225 4136 server_base.cc:1034] running on GCE node
I20260807 11:28:10.129966 4136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:10.131007 4136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:10.132273 4136 hybrid_clock.cc:648] HybridClock initialized: now 1786102090132213 us; error 49 us; skew 500 ppm
I20260807 11:28:10.132634 4136 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:10 dist-test-slave-2w3w krb5kdc[4022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102090, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:10.136183 4136 init.cc:377] Logged in from keytab as kudu/127.16.197.1@KRBTEST.COM (short username kudu)
I20260807 11:28:10.137754 4136 webserver.cc:460] Webserver started at http://127.16.197.1:36093/ using document root <none> and password file <none>
I20260807 11:28:10.138505 4136 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:10.138659 4136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:10.139065 4136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:10.142324 4136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-0/data/instance:
uuid: "4ea84ad7ecab45348dbcfb404b9a8d68"
format_stamp: "Formatted at 2026-08-07 11:28:10 on dist-test-slave-2w3w"
server_key: "3d91d162ad4518f2f8444fb16b601396"
server_key_iv: "5b8ae87a9c60b7f90e467677c71d6edd"
server_key_version: "encryptionkey@0"
I20260807 11:28:10.143229 4136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance:
uuid: "4ea84ad7ecab45348dbcfb404b9a8d68"
format_stamp: "Formatted at 2026-08-07 11:28:10 on dist-test-slave-2w3w"
server_key: "3d91d162ad4518f2f8444fb16b601396"
server_key_iv: "5b8ae87a9c60b7f90e467677c71d6edd"
server_key_version: "encryptionkey@0"
I20260807 11:28:10.148720 4136 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 11:28:10.152303 4152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:10.153702 4136 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 11:28:10.153905 4136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "4ea84ad7ecab45348dbcfb404b9a8d68"
format_stamp: "Formatted at 2026-08-07 11:28:10 on dist-test-slave-2w3w"
server_key: "3d91d162ad4518f2f8444fb16b601396"
server_key_iv: "5b8ae87a9c60b7f90e467677c71d6edd"
server_key_version: "encryptionkey@0"
I20260807 11:28:10.154129 4136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:10.166944 4136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:10.171432 4136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:10.171725 4136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:10.172684 4136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:28:10.174028 4136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:28:10.174134 4136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:10.174283 4136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:28:10.174358 4136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:10.208248 4136 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:37945
I20260807 11:28:10.208262 4271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:37945 every 8 connection(s)
I20260807 11:28:10.209661 4136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:28:10.211334 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 4136
I20260807 11:28:10.211536 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance
I20260807 11:28:10.211999 17172 external_mini_cluster.cc:1428] Setting key 17bbfb48876f32d8d26e659b414a39bc
Aug 07 11:28:10 dist-test-slave-2w3w krb5kdc[4022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102090, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:28:10.232604 4274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:10.212462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:36753 (local address 127.16.197.62:45631)
0807 11:28:10.213021 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:10.213028 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:10.215034 (+ 2006us) server_negotiation.cc:407] Connection header received
0807 11:28:10.216304 (+ 1270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:10.216310 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:10.216395 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:10.216548 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:10.218063 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:10.218874 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:10.220726 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:10.221477 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:10.221615 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:10.225399 (+ 3784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:10.225408 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:10.225415 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:10.225462 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:10.229164 (+ 3702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:10.229928 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:10.229936 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:10.229942 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:10.230091 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:10.230518 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:10.230525 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:10.230530 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:10.230942 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:10.231126 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:10.231820 (+ 694us) server_negotiation.cc:299] Negotiation successful
0807 11:28:10.232080 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":266,"threads_started":1}
I20260807 11:28:10.233213 4275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:10.213182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45631 (local address 127.16.197.1:36753)
0807 11:28:10.214069 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:10.214136 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:10.216036 (+ 1900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:10.216791 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:10.216800 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:10.217317 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:28:10.217887 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:10.217905 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:10.219057 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:10.219063 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:10.220543 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:10.220562 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:10.221716 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:10.221726 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:10.221983 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:10.222699 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:28:10.222723 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:28:10.225159 (+ 2436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:28:10.229316 (+ 4157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:28:10.229323 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:28:10.229337 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:28:10.229800 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:28:10.230204 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:28:10.230208 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:28:10.230210 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:28:10.230398 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:28:10.231089 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:28:10.231095 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:28:10.231434 (+ 339us) client_negotiation.cc:769] Sending connection context
0807 11:28:10.231674 (+ 240us) client_negotiation.cc:240] Negotiation successful
0807 11:28:10.232082 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":319,"threads_started":1}
I20260807 11:28:10.234812 4272 heartbeater.cc:347] Connected to a master server at 127.16.197.62:45631
I20260807 11:28:10.235160 4272 heartbeater.cc:464] Registering TS with master...
I20260807 11:28:10.235904 4272 heartbeater.cc:511] Master 127.16.197.62:45631 requested a full tablet report, sending...
I20260807 11:28:10.238179 4069 ts_manager.cc:194] Registered new tserver with Master: 4ea84ad7ecab45348dbcfb404b9a8d68 (127.16.197.1:37945)
I20260807 11:28:10.239711 4069 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.1@KRBTEST.COM'} at 127.16.197.1:36753
WARNING: no policy specified for kudu/127.16.197.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
I20260807 11:28:10.278182 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:0
--local_ip_for_outbound_sockets=127.16.197.2
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:45631
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:45523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:10.615974 4278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:10.616307 4278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:10.616472 4278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:10.623925 4278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:10.624096 4278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:10.624267 4278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:28:10.634392 4278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:45523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:45631
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:10.635790 4278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:10.637125 4278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:10.651553 4285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:10.652012 4288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:10.652060 4284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:10.652609 4278 server_base.cc:1034] running on GCE node
I20260807 11:28:10.653331 4278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:10.654325 4278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:10.655534 4278 hybrid_clock.cc:648] HybridClock initialized: now 1786102090655484 us; error 37 us; skew 500 ppm
I20260807 11:28:10.655874 4278 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:10 dist-test-slave-2w3w krb5kdc[4022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102090, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:10.662195 4278 init.cc:377] Logged in from keytab as kudu/127.16.197.2@KRBTEST.COM (short username kudu)
I20260807 11:28:10.663882 4278 webserver.cc:460] Webserver started at http://127.16.197.2:35669/ using document root <none> and password file <none>
I20260807 11:28:10.664597 4278 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:10.664741 4278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:10.665060 4278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:10.668066 4278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-1/data/instance:
uuid: "21ce74c9f19a4b248ae7454e6cc99d85"
format_stamp: "Formatted at 2026-08-07 11:28:10 on dist-test-slave-2w3w"
server_key: "77223975a800c5b62a1f81694ed88e5a"
server_key_iv: "68241e491bacf6abf6274981a23f5532"
server_key_version: "encryptionkey@0"
I20260807 11:28:10.668843 4278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance:
uuid: "21ce74c9f19a4b248ae7454e6cc99d85"
format_stamp: "Formatted at 2026-08-07 11:28:10 on dist-test-slave-2w3w"
server_key: "77223975a800c5b62a1f81694ed88e5a"
server_key_iv: "68241e491bacf6abf6274981a23f5532"
server_key_version: "encryptionkey@0"
I20260807 11:28:10.674463 4278 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 11:28:10.678227 4295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:10.679664 4278 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:28:10.679881 4278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "21ce74c9f19a4b248ae7454e6cc99d85"
format_stamp: "Formatted at 2026-08-07 11:28:10 on dist-test-slave-2w3w"
server_key: "77223975a800c5b62a1f81694ed88e5a"
server_key_iv: "68241e491bacf6abf6274981a23f5532"
server_key_version: "encryptionkey@0"
I20260807 11:28:10.680105 4278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:10.708639 4278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:10.712987 4278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:10.713294 4278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:10.714324 4278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:28:10.715705 4278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:28:10.715823 4278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:10.715975 4278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:28:10.716053 4278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:10.749801 4278 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:38683
I20260807 11:28:10.749858 4412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:38683 every 8 connection(s)
I20260807 11:28:10.751192 4278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:28:10.757015 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 4278
I20260807 11:28:10.757212 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance
I20260807 11:28:10.757815 17172 external_mini_cluster.cc:1428] Setting key 5d08135f822aef9c0035ab4364f2a470
Aug 07 11:28:10 dist-test-slave-2w3w krb5kdc[4022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102090, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:28:10.775117 4416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:10.753847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:52895 (local address 127.16.197.62:45631)
0807 11:28:10.754307 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:10.754311 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:10.755340 (+ 1029us) server_negotiation.cc:407] Connection header received
0807 11:28:10.757418 (+ 2078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:10.757423 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:10.757496 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:10.757597 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:10.759615 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:10.760377 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:10.763025 (+ 2648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:10.763479 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:10.763577 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:10.767904 (+ 4327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:10.767913 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:10.767921 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:10.767968 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:10.771612 (+ 3644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:10.772638 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:10.772643 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:10.772645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:10.772775 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:10.773286 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:10.773289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:10.773291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:10.773562 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:10.773671 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:10.774578 (+ 907us) server_negotiation.cc:299] Negotiation successful
0807 11:28:10.774777 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":237,"threads_started":1}
I20260807 11:28:10.776780 4417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:10.754447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45631 (local address 127.16.197.2:52895)
0807 11:28:10.755174 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:10.755216 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:10.757118 (+ 1902us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:10.757871 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:10.757949 (+ 78us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:10.758669 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:28:10.759446 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:10.759465 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:10.760534 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:10.760540 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:10.762810 (+ 2270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:10.762827 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:10.763716 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:10.763725 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:10.764066 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:10.764845 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:28:10.764879 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:28:10.767656 (+ 2777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:28:10.771759 (+ 4103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:28:10.771767 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:28:10.771781 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:28:10.772503 (+ 722us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:28:10.772885 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:28:10.772892 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:28:10.772897 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:28:10.773171 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:28:10.773686 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:28:10.773697 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:28:10.774181 (+ 484us) client_negotiation.cc:769] Sending connection context
0807 11:28:10.774499 (+ 318us) client_negotiation.cc:240] Negotiation successful
0807 11:28:10.775143 (+ 644us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":285,"threads_started":1}
I20260807 11:28:10.778314 4413 heartbeater.cc:347] Connected to a master server at 127.16.197.62:45631
I20260807 11:28:10.778633 4413 heartbeater.cc:464] Registering TS with master...
I20260807 11:28:10.779399 4413 heartbeater.cc:511] Master 127.16.197.62:45631 requested a full tablet report, sending...
I20260807 11:28:10.780937 4069 ts_manager.cc:194] Registered new tserver with Master: 21ce74c9f19a4b248ae7454e6cc99d85 (127.16.197.2:38683)
I20260807 11:28:10.781786 4069 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.2@KRBTEST.COM'} at 127.16.197.2:52895
WARNING: no policy specified for kudu/127.16.197.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
I20260807 11:28:10.824280 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:0
--local_ip_for_outbound_sockets=127.16.197.3
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:45631
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:45523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:11.178439 4420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:11.178774 4420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:11.178951 4420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:11.186426 4420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:11.186586 4420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:11.186767 4420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:28:11.196887 4420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:45523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:45631
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:11.198228 4420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:11.199637 4420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:11.214511 4426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:11.214534 4427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:11.214854 4429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:11.215703 4420 server_base.cc:1034] running on GCE node
I20260807 11:28:11.216281 4420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:11.217234 4420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:11.218429 4420 hybrid_clock.cc:648] HybridClock initialized: now 1786102091218372 us; error 51 us; skew 500 ppm
I20260807 11:28:11.218726 4420 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:11 dist-test-slave-2w3w krb5kdc[4022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102091, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:11.222050 4420 init.cc:377] Logged in from keytab as kudu/127.16.197.3@KRBTEST.COM (short username kudu)
I20260807 11:28:11.223500 4420 webserver.cc:460] Webserver started at http://127.16.197.3:43007/ using document root <none> and password file <none>
I20260807 11:28:11.224181 4420 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:11.224324 4420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:11.224654 4420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:11.227634 4420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-2/data/instance:
uuid: "bbb924177ea844a5927a5b2bdfa17270"
format_stamp: "Formatted at 2026-08-07 11:28:11 on dist-test-slave-2w3w"
server_key: "c7ff3230144dd0daf90bed19c92ead10"
server_key_iv: "06c139103c1c59bc19d51a070a69f522"
server_key_version: "encryptionkey@0"
I20260807 11:28:11.228475 4420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance:
uuid: "bbb924177ea844a5927a5b2bdfa17270"
format_stamp: "Formatted at 2026-08-07 11:28:11 on dist-test-slave-2w3w"
server_key: "c7ff3230144dd0daf90bed19c92ead10"
server_key_iv: "06c139103c1c59bc19d51a070a69f522"
server_key_version: "encryptionkey@0"
I20260807 11:28:11.233549 4420 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 11:28:11.236872 4436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:11.238106 4420 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 11:28:11.238270 4420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "bbb924177ea844a5927a5b2bdfa17270"
format_stamp: "Formatted at 2026-08-07 11:28:11 on dist-test-slave-2w3w"
server_key: "c7ff3230144dd0daf90bed19c92ead10"
server_key_iv: "06c139103c1c59bc19d51a070a69f522"
server_key_version: "encryptionkey@0"
I20260807 11:28:11.238484 4420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:11.242641 4272 heartbeater.cc:503] Master 127.16.197.62:45631 was elected leader, sending a full tablet report...
I20260807 11:28:11.270081 4420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:11.274269 4420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:11.274566 4420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:11.275629 4420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:28:11.276964 4420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:28:11.277060 4420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:11.277197 4420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:28:11.277272 4420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:11.310761 4420 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:45521
I20260807 11:28:11.310796 4554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:45521 every 8 connection(s)
I20260807 11:28:11.312129 4420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:28:11.322470 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 4420
I20260807 11:28:11.322666 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance
I20260807 11:28:11.323167 17172 external_mini_cluster.cc:1428] Setting key edd5181a3e67faf0d321c733e304873a
Aug 07 11:28:11 dist-test-slave-2w3w krb5kdc[4022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102091, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:28:11.332566 4557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:11.314674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:50907 (local address 127.16.197.62:45631)
0807 11:28:11.315189 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:11.315194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:11.316143 (+ 949us) server_negotiation.cc:407] Connection header received
0807 11:28:11.318024 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:11.318031 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:11.318137 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:11.318261 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:11.319852 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.320729 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.322280 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.322745 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.322897 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:11.327300 (+ 4403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:11.327305 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:11.327309 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:11.327353 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:11.329897 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:11.330567 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:11.330571 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:11.330573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:11.330698 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:11.331115 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:11.331118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:11.331120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:11.331371 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:11.331499 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:11.332124 (+ 625us) server_negotiation.cc:299] Negotiation successful
0807 11:28:11.332294 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":255,"threads_started":1}
I20260807 11:28:11.333518 4558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:11.315289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:45631 (local address 127.16.197.3:50907)
0807 11:28:11.315979 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:11.316020 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:11.317745 (+ 1725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:11.318509 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:11.318517 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:11.319094 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:28:11.319663 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:11.319680 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.320879 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.320889 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:11.322137 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:11.322150 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.323016 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.323027 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:11.323301 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:11.324069 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:28:11.324098 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:28:11.327117 (+ 3019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:28:11.330036 (+ 2919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:28:11.330043 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:28:11.330056 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:28:11.330463 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:28:11.330783 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:28:11.330786 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:28:11.330789 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:28:11.331026 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:28:11.331470 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:28:11.331476 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:28:11.331781 (+ 305us) client_negotiation.cc:769] Sending connection context
0807 11:28:11.332020 (+ 239us) client_negotiation.cc:240] Negotiation successful
0807 11:28:11.332400 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":292,"threads_started":1}
I20260807 11:28:11.334913 4555 heartbeater.cc:347] Connected to a master server at 127.16.197.62:45631
I20260807 11:28:11.335227 4555 heartbeater.cc:464] Registering TS with master...
I20260807 11:28:11.335892 4555 heartbeater.cc:511] Master 127.16.197.62:45631 requested a full tablet report, sending...
I20260807 11:28:11.337373 4069 ts_manager.cc:194] Registered new tserver with Master: bbb924177ea844a5927a5b2bdfa17270 (127.16.197.3:45521)
I20260807 11:28:11.338497 4069 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.3@KRBTEST.COM'} at 127.16.197.3:50907
I20260807 11:28:11.338793 17172 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 11:28:11.355430 4557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:11.342798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56484 (local address 127.16.197.62:45631)
0807 11:28:11.343006 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:11.343014 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:11.343492 (+ 478us) server_negotiation.cc:407] Connection header received
0807 11:28:11.343620 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:11.343623 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:11.343682 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:11.343762 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:11.345251 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.345946 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.347292 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.347687 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.347778 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:11.350261 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:11.350266 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:11.350269 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:11.350298 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:11.352945 (+ 2647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:11.353596 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:11.353600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:11.353602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:11.353707 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:11.354066 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:11.354069 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:11.354071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:11.354374 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:11.354577 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:11.354876 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 11:28:11.355069 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 11:28:11.359323 4069 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56484:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 11:28:11.366165 4069 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:28:11.401575 4570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:11.377924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:38683 (local address 127.0.0.1:58620)
0807 11:28:11.390114 (+ 12190us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:11.390177 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:11.390441 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:11.392585 (+ 2144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:11.392597 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:11.392642 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:11.393211 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:11.393228 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.396170 (+ 2942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.396197 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:11.398806 (+ 2609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:11.398822 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.400525 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.400530 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:11.400661 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:11.400693 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 11:28:11.400841 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 11:28:11.401083 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11757,"mutex_wait_us":1,"thread_start_us":16655,"threads_started":1}
I20260807 11:28:11.402555 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:11.378143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58620 (local address 127.16.197.2:38683)
0807 11:28:11.391443 (+ 13300us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:11.391458 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:11.391501 (+ 43us) server_negotiation.cc:407] Connection header received
0807 11:28:11.391631 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:11.391643 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:11.391973 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:11.392220 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:11.393711 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.395971 (+ 2260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.399032 (+ 3061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.400138 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.400222 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:11.400368 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:11.402057 (+ 1689us) server_negotiation.cc:299] Negotiation successful
0807 11:28:11.402228 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13067,"thread_start_us":15257,"threads_started":1}
I20260807 11:28:11.404366 4569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:11.378099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:45521 (local address 127.0.0.1:52130)
0807 11:28:11.394861 (+ 16762us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:11.394880 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:11.395039 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:11.396976 (+ 1937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:11.396984 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:11.397014 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:11.397528 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:11.397546 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.399497 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.399502 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:11.401309 (+ 1807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:11.401320 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.403655 (+ 2335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.403660 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:11.403775 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:11.403798 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 11:28:11.403897 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 11:28:11.404022 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16646,"thread_start_us":11416,"threads_started":1}
I20260807 11:28:11.404449 4573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:11.378438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52130 (local address 127.16.197.3:45521)
0807 11:28:11.394178 (+ 15740us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:11.394186 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:11.396036 (+ 1850us) server_negotiation.cc:407] Connection header received
0807 11:28:11.396175 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:11.396203 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:11.396478 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:11.396676 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:11.397766 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.399337 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.401468 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.403245 (+ 1777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.403368 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:11.403544 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:11.403965 (+ 421us) server_negotiation.cc:299] Negotiation successful
0807 11:28:11.404123 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15559,"thread_start_us":15395,"threads_started":1}
I20260807 11:28:11.405881 4575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:11.394238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:37945 (local address 127.0.0.1:44062)
0807 11:28:11.397046 (+ 2808us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:11.397068 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:11.397217 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:11.397775 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:11.397779 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:11.397803 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:11.398866 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:11.398881 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.400851 (+ 1970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.400856 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:11.403024 (+ 2168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:11.403040 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.405026 (+ 1986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.405033 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:11.405198 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:11.405231 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 11:28:11.405324 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 11:28:11.405472 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2646,"mutex_wait_us":1,"thread_start_us":2505,"threads_started":1}
I20260807 11:28:11.405927 4574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:11.391070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44062 (local address 127.16.197.1:37945)
0807 11:28:11.396726 (+ 5656us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:11.396736 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:11.397188 (+ 452us) server_negotiation.cc:407] Connection header received
0807 11:28:11.397383 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:11.397391 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:11.397610 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:11.397793 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:11.399154 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.400698 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.403208 (+ 2510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.404812 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.404966 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:11.405202 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:11.405427 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 11:28:11.405606 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5476,"thread_start_us":5318,"threads_started":1}
I20260807 11:28:11.407387 4345 tablet_service.cc:1467] Processing CreateTablet for tablet 54e2f59fa92645ee93083cff45d5099d (DEFAULT_TABLE table=test-table [id=1ffaf3de26f4433d8f3b3f066903f8e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:28:11.408103 4487 tablet_service.cc:1467] Processing CreateTablet for tablet 54e2f59fa92645ee93083cff45d5099d (DEFAULT_TABLE table=test-table [id=1ffaf3de26f4433d8f3b3f066903f8e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:28:11.408610 4345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54e2f59fa92645ee93083cff45d5099d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:11.409212 4487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54e2f59fa92645ee93083cff45d5099d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:11.413689 4203 tablet_service.cc:1467] Processing CreateTablet for tablet 54e2f59fa92645ee93083cff45d5099d (DEFAULT_TABLE table=test-table [id=1ffaf3de26f4433d8f3b3f066903f8e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:28:11.415343 4203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54e2f59fa92645ee93083cff45d5099d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:11.420617 4576 tablet_bootstrap.cc:492] T 54e2f59fa92645ee93083cff45d5099d P 21ce74c9f19a4b248ae7454e6cc99d85: Bootstrap starting.
I20260807 11:28:11.424259 4576 tablet_bootstrap.cc:654] T 54e2f59fa92645ee93083cff45d5099d P 21ce74c9f19a4b248ae7454e6cc99d85: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:11.424819 4577 tablet_bootstrap.cc:492] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270: Bootstrap starting.
I20260807 11:28:11.425756 4576 log.cc:824] T 54e2f59fa92645ee93083cff45d5099d P 21ce74c9f19a4b248ae7454e6cc99d85: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:11.428505 4577 tablet_bootstrap.cc:654] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:11.429275 4576 tablet_bootstrap.cc:492] T 54e2f59fa92645ee93083cff45d5099d P 21ce74c9f19a4b248ae7454e6cc99d85: No bootstrap required, opened a new log
I20260807 11:28:11.429685 4576 ts_tablet_manager.cc:1397] T 54e2f59fa92645ee93083cff45d5099d P 21ce74c9f19a4b248ae7454e6cc99d85: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260807 11:28:11.429764 4580 tablet_bootstrap.cc:492] T 54e2f59fa92645ee93083cff45d5099d P 4ea84ad7ecab45348dbcfb404b9a8d68: Bootstrap starting.
I20260807 11:28:11.429823 4577 log.cc:824] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:11.433477 4580 tablet_bootstrap.cc:654] T 54e2f59fa92645ee93083cff45d5099d P 4ea84ad7ecab45348dbcfb404b9a8d68: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:11.434376 4577 tablet_bootstrap.cc:492] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270: No bootstrap required, opened a new log
I20260807 11:28:11.434672 4577 ts_tablet_manager.cc:1397] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 11:28:11.434783 4580 log.cc:824] T 54e2f59fa92645ee93083cff45d5099d P 4ea84ad7ecab45348dbcfb404b9a8d68: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:11.437861 4580 tablet_bootstrap.cc:492] T 54e2f59fa92645ee93083cff45d5099d P 4ea84ad7ecab45348dbcfb404b9a8d68: No bootstrap required, opened a new log
I20260807 11:28:11.438128 4580 ts_tablet_manager.cc:1397] T 54e2f59fa92645ee93083cff45d5099d P 4ea84ad7ecab45348dbcfb404b9a8d68: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260807 11:28:11.438323 4576 raft_consensus.cc:348] T 54e2f59fa92645ee93083cff45d5099d P 21ce74c9f19a4b248ae7454e6cc99d85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21ce74c9f19a4b248ae7454e6cc99d85" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38683 } } peers { permanent_uuid: "bbb924177ea844a5927a5b2bdfa17270" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45521 } } peers { permanent_uuid: "4ea84ad7ecab45348dbcfb404b9a8d68" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 37945 } }
I20260807 11:28:11.438694 4576 raft_consensus.cc:374] T 54e2f59fa92645ee93083cff45d5099d P 21ce74c9f19a4b248ae7454e6cc99d85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:11.438822 4576 raft_consensus.cc:729] T 54e2f59fa92645ee93083cff45d5099d P 21ce74c9f19a4b248ae7454e6cc99d85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21ce74c9f19a4b248ae7454e6cc99d85, State: Initialized, Role: FOLLOWER
I20260807 11:28:11.439479 4576 consensus_queue.cc:260] T 54e2f59fa92645ee93083cff45d5099d P 21ce74c9f19a4b248ae7454e6cc99d85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21ce74c9f19a4b248ae7454e6cc99d85" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38683 } } peers { permanent_uuid: "bbb924177ea844a5927a5b2bdfa17270" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45521 } } peers { permanent_uuid: "4ea84ad7ecab45348dbcfb404b9a8d68" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 37945 } }
I20260807 11:28:11.445278 4577 raft_consensus.cc:348] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21ce74c9f19a4b248ae7454e6cc99d85" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38683 } } peers { permanent_uuid: "bbb924177ea844a5927a5b2bdfa17270" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45521 } } peers { permanent_uuid: "4ea84ad7ecab45348dbcfb404b9a8d68" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 37945 } }
I20260807 11:28:11.445719 4577 raft_consensus.cc:374] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:11.445818 4577 raft_consensus.cc:729] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbb924177ea844a5927a5b2bdfa17270, State: Initialized, Role: FOLLOWER
I20260807 11:28:11.446415 4577 consensus_queue.cc:260] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21ce74c9f19a4b248ae7454e6cc99d85" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38683 } } peers { permanent_uuid: "bbb924177ea844a5927a5b2bdfa17270" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45521 } } peers { permanent_uuid: "4ea84ad7ecab45348dbcfb404b9a8d68" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 37945 } }
I20260807 11:28:11.446823 4413 heartbeater.cc:503] Master 127.16.197.62:45631 was elected leader, sending a full tablet report...
I20260807 11:28:11.447857 4576 ts_tablet_manager.cc:1428] T 54e2f59fa92645ee93083cff45d5099d P 21ce74c9f19a4b248ae7454e6cc99d85: Time spent starting tablet: real 0.018s user 0.016s sys 0.000s
I20260807 11:28:11.448746 4555 heartbeater.cc:503] Master 127.16.197.62:45631 was elected leader, sending a full tablet report...
I20260807 11:28:11.449330 4577 ts_tablet_manager.cc:1428] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260807 11:28:11.449120 4580 raft_consensus.cc:348] T 54e2f59fa92645ee93083cff45d5099d P 4ea84ad7ecab45348dbcfb404b9a8d68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21ce74c9f19a4b248ae7454e6cc99d85" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38683 } } peers { permanent_uuid: "bbb924177ea844a5927a5b2bdfa17270" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45521 } } peers { permanent_uuid: "4ea84ad7ecab45348dbcfb404b9a8d68" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 37945 } }
I20260807 11:28:11.449550 4580 raft_consensus.cc:374] T 54e2f59fa92645ee93083cff45d5099d P 4ea84ad7ecab45348dbcfb404b9a8d68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:11.449656 4580 raft_consensus.cc:729] T 54e2f59fa92645ee93083cff45d5099d P 4ea84ad7ecab45348dbcfb404b9a8d68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ea84ad7ecab45348dbcfb404b9a8d68, State: Initialized, Role: FOLLOWER
I20260807 11:28:11.450341 4580 consensus_queue.cc:260] T 54e2f59fa92645ee93083cff45d5099d P 4ea84ad7ecab45348dbcfb404b9a8d68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21ce74c9f19a4b248ae7454e6cc99d85" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38683 } } peers { permanent_uuid: "bbb924177ea844a5927a5b2bdfa17270" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45521 } } peers { permanent_uuid: "4ea84ad7ecab45348dbcfb404b9a8d68" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 37945 } }
I20260807 11:28:11.453025 4580 ts_tablet_manager.cc:1428] T 54e2f59fa92645ee93083cff45d5099d P 4ea84ad7ecab45348dbcfb404b9a8d68: Time spent starting tablet: real 0.015s user 0.010s sys 0.005s
W20260807 11:28:11.462124 4273 tablet.cc:2369] T 54e2f59fa92645ee93083cff45d5099d P 4ea84ad7ecab45348dbcfb404b9a8d68: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 11:28:11.503304 4414 tablet.cc:2369] T 54e2f59fa92645ee93083cff45d5099d P 21ce74c9f19a4b248ae7454e6cc99d85: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 11:28:11.564077 4556 tablet.cc:2369] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:28:11.592831 4584 raft_consensus.cc:482] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:28:11.593218 4584 raft_consensus.cc:504] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21ce74c9f19a4b248ae7454e6cc99d85" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38683 } } peers { permanent_uuid: "bbb924177ea844a5927a5b2bdfa17270" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45521 } } peers { permanent_uuid: "4ea84ad7ecab45348dbcfb404b9a8d68" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 37945 } }
I20260807 11:28:11.595062 4584 leader_election.cc:288] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21ce74c9f19a4b248ae7454e6cc99d85 (127.16.197.2:38683), 4ea84ad7ecab45348dbcfb404b9a8d68 (127.16.197.1:37945)
I20260807 11:28:11.601552 4558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:11.595530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:38683 (local address 127.16.197.3:41225)
0807 11:28:11.595729 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:11.595780 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:11.595931 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:11.596268 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:11.596273 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:11.596302 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:11.596505 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:11.596515 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.597728 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.597731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:11.599592 (+ 1861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:11.599608 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.600954 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.600959 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:11.601117 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:11.601143 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 11:28:11.601229 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 11:28:11.601338 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 11:28:11.601713 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:11.595538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:41225 (local address 127.16.197.2:38683)
0807 11:28:11.595770 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:11.595778 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:11.595850 (+ 72us) server_negotiation.cc:407] Connection header received
0807 11:28:11.596072 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:11.596076 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:11.596136 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:11.596251 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:11.596651 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.597607 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.599766 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.600783 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.600956 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:11.601037 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:11.601284 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 11:28:11.601442 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 11:28:11.602536 4586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:11.595530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:37945 (local address 127.16.197.3:34671)
0807 11:28:11.596592 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:11.596610 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:11.596698 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:11.596979 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:11.596982 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:11.596998 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:11.597290 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:11.597296 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.598423 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.598429 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:11.600187 (+ 1758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:11.600197 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.601741 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.601747 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:11.601949 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:11.601979 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 11:28:11.602062 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 11:28:11.602185 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":944,"mutex_wait_us":244,"thread_start_us":446,"threads_started":1}
I20260807 11:28:11.602739 4367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54e2f59fa92645ee93083cff45d5099d" candidate_uuid: "bbb924177ea844a5927a5b2bdfa17270" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21ce74c9f19a4b248ae7454e6cc99d85" is_pre_election: true
I20260807 11:28:11.603093 4574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:11.595613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:34671 (local address 127.16.197.1:37945)
0807 11:28:11.595819 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:11.595825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:11.596698 (+ 873us) server_negotiation.cc:407] Connection header received
0807 11:28:11.596787 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:11.596792 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:11.596868 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:11.596971 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:11.597441 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.598256 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.600338 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.601508 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.601665 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:11.601745 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:11.602115 (+ 370us) server_negotiation.cc:299] Negotiation successful
0807 11:28:11.602819 (+ 704us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 11:28:11.603387 4367 raft_consensus.cc:2454] T 54e2f59fa92645ee93083cff45d5099d P 21ce74c9f19a4b248ae7454e6cc99d85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbb924177ea844a5927a5b2bdfa17270 in term 0.
I20260807 11:28:11.604169 4438 leader_election.cc:302] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21ce74c9f19a4b248ae7454e6cc99d85, bbb924177ea844a5927a5b2bdfa17270; no voters:
I20260807 11:28:11.604113 4226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54e2f59fa92645ee93083cff45d5099d" candidate_uuid: "bbb924177ea844a5927a5b2bdfa17270" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ea84ad7ecab45348dbcfb404b9a8d68" is_pre_election: true
I20260807 11:28:11.604590 4584 raft_consensus.cc:2781] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:28:11.604635 4226 raft_consensus.cc:2454] T 54e2f59fa92645ee93083cff45d5099d P 4ea84ad7ecab45348dbcfb404b9a8d68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbb924177ea844a5927a5b2bdfa17270 in term 0.
I20260807 11:28:11.604760 4584 raft_consensus.cc:482] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:28:11.604869 4584 raft_consensus.cc:3037] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:11.607203 4584 raft_consensus.cc:504] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21ce74c9f19a4b248ae7454e6cc99d85" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38683 } } peers { permanent_uuid: "bbb924177ea844a5927a5b2bdfa17270" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45521 } } peers { permanent_uuid: "4ea84ad7ecab45348dbcfb404b9a8d68" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 37945 } }
I20260807 11:28:11.607822 4584 leader_election.cc:288] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270 [CANDIDATE]: Term 1 election: Requested vote from peers 21ce74c9f19a4b248ae7454e6cc99d85 (127.16.197.2:38683), 4ea84ad7ecab45348dbcfb404b9a8d68 (127.16.197.1:37945)
I20260807 11:28:11.608407 4226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54e2f59fa92645ee93083cff45d5099d" candidate_uuid: "bbb924177ea844a5927a5b2bdfa17270" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ea84ad7ecab45348dbcfb404b9a8d68"
I20260807 11:28:11.608410 4367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54e2f59fa92645ee93083cff45d5099d" candidate_uuid: "bbb924177ea844a5927a5b2bdfa17270" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21ce74c9f19a4b248ae7454e6cc99d85"
I20260807 11:28:11.608633 4367 raft_consensus.cc:3037] T 54e2f59fa92645ee93083cff45d5099d P 21ce74c9f19a4b248ae7454e6cc99d85 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:11.608634 4226 raft_consensus.cc:3037] T 54e2f59fa92645ee93083cff45d5099d P 4ea84ad7ecab45348dbcfb404b9a8d68 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:11.611902 4226 raft_consensus.cc:2454] T 54e2f59fa92645ee93083cff45d5099d P 4ea84ad7ecab45348dbcfb404b9a8d68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbb924177ea844a5927a5b2bdfa17270 in term 1.
I20260807 11:28:11.611902 4367 raft_consensus.cc:2454] T 54e2f59fa92645ee93083cff45d5099d P 21ce74c9f19a4b248ae7454e6cc99d85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbb924177ea844a5927a5b2bdfa17270 in term 1.
I20260807 11:28:11.612396 4439 leader_election.cc:302] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ea84ad7ecab45348dbcfb404b9a8d68, bbb924177ea844a5927a5b2bdfa17270; no voters:
I20260807 11:28:11.612712 4584 raft_consensus.cc:2781] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:11.613154 4584 raft_consensus.cc:686] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270 [term 1 LEADER]: Becoming Leader. State: Replica: bbb924177ea844a5927a5b2bdfa17270, State: Running, Role: LEADER
I20260807 11:28:11.613619 4584 consensus_queue.cc:237] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21ce74c9f19a4b248ae7454e6cc99d85" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38683 } } peers { permanent_uuid: "bbb924177ea844a5927a5b2bdfa17270" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45521 } } peers { permanent_uuid: "4ea84ad7ecab45348dbcfb404b9a8d68" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 37945 } }
I20260807 11:28:11.618050 4067 catalog_manager.cc:5495] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270 reported cstate change: term changed from 0 to 1, leader changed from <none> to bbb924177ea844a5927a5b2bdfa17270 (127.16.197.3). New cstate: current_term: 1 leader_uuid: "bbb924177ea844a5927a5b2bdfa17270" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21ce74c9f19a4b248ae7454e6cc99d85" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbb924177ea844a5927a5b2bdfa17270" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ea84ad7ecab45348dbcfb404b9a8d68" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 37945 } health_report { overall_health: UNKNOWN } } }
I20260807 11:28:11.652338 4573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:11.647070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52134 (local address 127.16.197.3:45521)
0807 11:28:11.647259 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:11.647264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:11.647329 (+ 65us) server_negotiation.cc:407] Connection header received
0807 11:28:11.647535 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:11.647539 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:11.647619 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:11.647791 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:28:11.648113 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.648860 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.650293 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.650702 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.650833 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:11.651308 (+ 475us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:28:11.651857 (+ 549us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:28:11.651992 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:11.652088 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 11:28:11.652129 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 11:28:11.661115 4226 raft_consensus.cc:1261] T 54e2f59fa92645ee93083cff45d5099d P 4ea84ad7ecab45348dbcfb404b9a8d68 [term 1 FOLLOWER]: Refusing update from remote peer bbb924177ea844a5927a5b2bdfa17270: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:28:11.661115 4367 raft_consensus.cc:1261] T 54e2f59fa92645ee93083cff45d5099d P 21ce74c9f19a4b248ae7454e6cc99d85 [term 1 FOLLOWER]: Refusing update from remote peer bbb924177ea844a5927a5b2bdfa17270: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:28:11.662252 4587 consensus_queue.cc:1035] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21ce74c9f19a4b248ae7454e6cc99d85" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 38683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:28:11.662632 4584 consensus_queue.cc:1035] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ea84ad7ecab45348dbcfb404b9a8d68" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 37945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:28:11.674319 4590 mvcc.cc:205] Tried to move back new op lower bound from 7315874167432015872 to 7315874167252914176. Current Snapshot: MvccSnapshot[applied={T|T < 7315874167432015872}]
I20260807 11:28:11.675664 4592 mvcc.cc:205] Tried to move back new op lower bound from 7315874167432015872 to 7315874167252914176. Current Snapshot: MvccSnapshot[applied={T|T < 7315874167432015872}]
I20260807 11:28:11.690443 4067 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:28:11.690725 4067 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:28:11.694381 4067 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5dfddbdb0c9d45b5ac0f176a3bc896f3: Sending DeleteTablet for 3 replicas of tablet 54e2f59fa92645ee93083cff45d5099d
I20260807 11:28:11.695662 4345 tablet_service.cc:1514] Processing DeleteTablet for tablet 54e2f59fa92645ee93083cff45d5099d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:28:11 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:58620
I20260807 11:28:11.695979 4487 tablet_service.cc:1514] Processing DeleteTablet for tablet 54e2f59fa92645ee93083cff45d5099d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:28:11 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:52130
I20260807 11:28:11.696132 4203 tablet_service.cc:1514] Processing DeleteTablet for tablet 54e2f59fa92645ee93083cff45d5099d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:28:11 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:44062
I20260807 11:28:11.697357 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 4136
I20260807 11:28:11.706998 4601 tablet_replica.cc:330] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270: stopping tablet replica
I20260807 11:28:11.707667 4601 raft_consensus.cc:2229] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270 [term 1 LEADER]: Raft consensus shutting down.
I20260807 11:28:11.708499 4601 raft_consensus.cc:2258] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:28:11.709147 4600 tablet_replica.cc:330] T 54e2f59fa92645ee93083cff45d5099d P 21ce74c9f19a4b248ae7454e6cc99d85: stopping tablet replica
I20260807 11:28:11.709803 4602 tablet_replica.cc:330] T 54e2f59fa92645ee93083cff45d5099d P 4ea84ad7ecab45348dbcfb404b9a8d68: stopping tablet replica
I20260807 11:28:11.710335 4600 raft_consensus.cc:2229] T 54e2f59fa92645ee93083cff45d5099d P 21ce74c9f19a4b248ae7454e6cc99d85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:28:11.710862 4602 raft_consensus.cc:2229] T 54e2f59fa92645ee93083cff45d5099d P 4ea84ad7ecab45348dbcfb404b9a8d68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:28:11.711488 4600 raft_consensus.cc:2258] T 54e2f59fa92645ee93083cff45d5099d P 21ce74c9f19a4b248ae7454e6cc99d85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:28:11.712234 4602 raft_consensus.cc:2258] T 54e2f59fa92645ee93083cff45d5099d P 4ea84ad7ecab45348dbcfb404b9a8d68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:28:11.712543 4601 ts_tablet_manager.cc:1905] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:28:11.714160 4600 ts_tablet_manager.cc:1905] T 54e2f59fa92645ee93083cff45d5099d P 21ce74c9f19a4b248ae7454e6cc99d85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:28:11.720573 4601 ts_tablet_manager.cc:1918] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:28:11.720767 4601 log.cc:1196] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-2/wal/wals/54e2f59fa92645ee93083cff45d5099d
Aug 07 11:28:11 dist-test-slave-2w3w krb5kdc[4022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.1@KRBTEST.COM
I20260807 11:28:11.720964 4602 ts_tablet_manager.cc:1905] T 54e2f59fa92645ee93083cff45d5099d P 4ea84ad7ecab45348dbcfb404b9a8d68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:28:11.721323 4601 ts_tablet_manager.cc:1939] T 54e2f59fa92645ee93083cff45d5099d P bbb924177ea844a5927a5b2bdfa17270: Deleting consensus metadata
I20260807 11:28:11.723225 4055 catalog_manager.cc:4841] TS bbb924177ea844a5927a5b2bdfa17270 (127.16.197.3:45521): tablet 54e2f59fa92645ee93083cff45d5099d (table test-table [id=1ffaf3de26f4433d8f3b3f066903f8e0]) successfully deleted
I20260807 11:28:11.726240 4600 ts_tablet_manager.cc:1918] T 54e2f59fa92645ee93083cff45d5099d P 21ce74c9f19a4b248ae7454e6cc99d85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:28:11.726475 4600 log.cc:1196] T 54e2f59fa92645ee93083cff45d5099d P 21ce74c9f19a4b248ae7454e6cc99d85: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-1/wal/wals/54e2f59fa92645ee93083cff45d5099d
I20260807 11:28:11.727080 4600 ts_tablet_manager.cc:1939] T 54e2f59fa92645ee93083cff45d5099d P 21ce74c9f19a4b248ae7454e6cc99d85: Deleting consensus metadata
I20260807 11:28:11.728281 4602 ts_tablet_manager.cc:1918] T 54e2f59fa92645ee93083cff45d5099d P 4ea84ad7ecab45348dbcfb404b9a8d68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:28:11.728454 4602 log.cc:1196] T 54e2f59fa92645ee93083cff45d5099d P 4ea84ad7ecab45348dbcfb404b9a8d68: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786101765888975-17172-0/minicluster-data/ts-0/wal/wals/54e2f59fa92645ee93083cff45d5099d
I20260807 11:28:11.729027 4602 ts_tablet_manager.cc:1939] T 54e2f59fa92645ee93083cff45d5099d P 4ea84ad7ecab45348dbcfb404b9a8d68: Deleting consensus metadata
I20260807 11:28:11.729058 4054 catalog_manager.cc:4841] TS 21ce74c9f19a4b248ae7454e6cc99d85 (127.16.197.2:38683): tablet 54e2f59fa92645ee93083cff45d5099d (table test-table [id=1ffaf3de26f4433d8f3b3f066903f8e0]) successfully deleted
I20260807 11:28:11.730791 4574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:11.698665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44070 (local address 127.16.197.1:37945)
0807 11:28:11.698918 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:11.698927 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:11.710108 (+ 11181us) server_negotiation.cc:407] Connection header received
0807 11:28:11.710327 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:11.710333 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:11.710416 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:11.710527 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:11.712266 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.713405 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.715531 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.716228 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.716332 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:11.722466 (+ 6134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:11.722481 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:11.722498 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:11.722542 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:11.726953 (+ 4411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:11.727886 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:11.727896 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:11.727899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:11.728089 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:11.728800 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:11.728806 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:11.728809 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:11.729409 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:11.729756 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:11.730126 (+ 370us) server_negotiation.cc:299] Negotiation successful
0807 11:28:11.730399 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 11:28:11.730816 4055 catalog_manager.cc:4841] TS 4ea84ad7ecab45348dbcfb404b9a8d68 (127.16.197.1:37945): tablet 54e2f59fa92645ee93083cff45d5099d (table test-table [id=1ffaf3de26f4433d8f3b3f066903f8e0]) successfully deleted
I20260807 11:28:11.731563 4267 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:11.851246 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 4136
I20260807 11:28:11.878546 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 4278
Aug 07 11:28:11 dist-test-slave-2w3w krb5kdc[4022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.2@KRBTEST.COM
I20260807 11:28:11.896660 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:11.879497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58632 (local address 127.16.197.2:38683)
0807 11:28:11.879737 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:11.879745 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:11.879768 (+ 23us) server_negotiation.cc:407] Connection header received
0807 11:28:11.879847 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:11.879852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:11.879926 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:11.880031 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:11.881378 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.882167 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.883794 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:11.884448 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:11.884609 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:11.889238 (+ 4629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:11.889259 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:11.889281 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:11.889351 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:11.893400 (+ 4049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:11.894149 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:11.894159 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:11.894163 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:11.894341 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:11.894702 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:11.894707 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:11.894712 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:11.895247 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:11.895565 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:11.896008 (+ 443us) server_negotiation.cc:299] Negotiation successful
0807 11:28:11.896241 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260807 11:28:11.897377 4408 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:11.989616 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 4278
I20260807 11:28:12.010691 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 4420
Aug 07 11:28:12 dist-test-slave-2w3w krb5kdc[4022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.3@KRBTEST.COM
I20260807 11:28:12.029896 4573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:12.011642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52138 (local address 127.16.197.3:45521)
0807 11:28:12.011858 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:12.011865 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:12.011891 (+ 26us) server_negotiation.cc:407] Connection header received
0807 11:28:12.012026 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:12.012032 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:12.012120 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:12.012256 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:12.013823 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:12.015299 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:12.016768 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:12.017203 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:12.017352 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:12.022488 (+ 5136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:12.022500 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:12.022516 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:12.022562 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:12.025887 (+ 3325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:12.026835 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:12.026842 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:12.026844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:12.027307 (+ 463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:12.027752 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:12.027755 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:12.027757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:12.028131 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:12.028353 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:12.028917 (+ 564us) server_negotiation.cc:299] Negotiation successful
0807 11:28:12.029355 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 11:28:12.030553 4550 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:12.154421 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 4420
I20260807 11:28:12.175632 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 4035
I20260807 11:28:12.176712 4102 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:12.289803 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 4035
2026-08-07T11:28:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4617 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T11:28:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:28:12Z Disabled control of system clock
I20260807 11:28:12.374679 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:41205
--webserver_interface=127.16.197.62
--webserver_port=0
--builtin_ntp_servers=127.16.197.20:42177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:41205
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 11:28:12.711830 4611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:12.712132 4611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:12.712241 4611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:12.719677 4611 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:28:12.719795 4611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:12.719834 4611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:12.719892 4611 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:28:12.719939 4611 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:28:12.729066 4611 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:42177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:41205
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:41205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:12.730592 4611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:12.732589 4611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:12.744645 4621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:12.744671 4618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:12.744730 4619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:12.745720 4611 server_base.cc:1034] running on GCE node
I20260807 11:28:12.746472 4611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:12.747949 4611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:12.749246 4611 hybrid_clock.cc:648] HybridClock initialized: now 1786102092749188 us; error 49 us; skew 500 ppm
I20260807 11:28:12.749636 4611 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 11:28:12.752004 4611 webserver.cc:460] Webserver started at http://127.16.197.62:41919/ using document root <none> and password file <none>
I20260807 11:28:12.752669 4611 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:12.752777 4611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:12.753109 4611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:12.756099 4611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "94ae53a5c8dc4fc0b6ab01b61bd33557"
format_stamp: "Formatted at 2026-08-07 11:28:12 on dist-test-slave-2w3w"
server_key: "e3188bd507a28272827d01483452025c"
server_key_iv: "5e164a8d38ab2cce10ed682b3c88a454"
server_key_version: "encryptionkey@0"
I20260807 11:28:12.756937 4611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "94ae53a5c8dc4fc0b6ab01b61bd33557"
format_stamp: "Formatted at 2026-08-07 11:28:12 on dist-test-slave-2w3w"
server_key: "e3188bd507a28272827d01483452025c"
server_key_iv: "5e164a8d38ab2cce10ed682b3c88a454"
server_key_version: "encryptionkey@0"
I20260807 11:28:12.762225 4611 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 11:28:12.765688 4627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:12.767243 4611 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:28:12.767445 4611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "94ae53a5c8dc4fc0b6ab01b61bd33557"
format_stamp: "Formatted at 2026-08-07 11:28:12 on dist-test-slave-2w3w"
server_key: "e3188bd507a28272827d01483452025c"
server_key_iv: "5e164a8d38ab2cce10ed682b3c88a454"
server_key_version: "encryptionkey@0"
I20260807 11:28:12.767681 4611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:12.808765 4611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:12.809719 4611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:12.810039 4611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:12.829324 4611 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:41205
I20260807 11:28:12.829344 4682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:41205 every 8 connection(s)
I20260807 11:28:12.830819 4611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:28:12.835311 4683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:12.840422 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 4611
I20260807 11:28:12.840598 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:28:12.841033 17172 external_mini_cluster.cc:1428] Setting key c932a1ff2d88a858a8572b621e782876
I20260807 11:28:12.847285 4683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94ae53a5c8dc4fc0b6ab01b61bd33557: Bootstrap starting.
I20260807 11:28:12.851255 4683 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94ae53a5c8dc4fc0b6ab01b61bd33557: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:12.852476 4683 log.cc:824] T 00000000000000000000000000000000 P 94ae53a5c8dc4fc0b6ab01b61bd33557: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:12.856133 4683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94ae53a5c8dc4fc0b6ab01b61bd33557: No bootstrap required, opened a new log
I20260807 11:28:12.857390 4686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:12.842997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55876 (local address 127.16.197.62:41205)
0807 11:28:12.844187 (+ 1190us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:12.844202 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:12.844258 (+ 56us) server_negotiation.cc:407] Connection header received
0807 11:28:12.846691 (+ 2433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:12.846718 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:12.847346 (+ 628us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:12.847786 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:12.848327 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:12.850417 (+ 2090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:12.851879 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:12.852957 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:12.853184 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:12.854458 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:12.854470 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:12.854488 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:12.855277 (+ 789us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:12.855387 (+ 110us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:12.855424 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:12.855548 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:12.855728 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 11:28:12.856059 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":396,"threads_started":1}
I20260807 11:28:12.864068 4683 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94ae53a5c8dc4fc0b6ab01b61bd33557 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94ae53a5c8dc4fc0b6ab01b61bd33557" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 41205 } }
I20260807 11:28:12.864449 4683 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94ae53a5c8dc4fc0b6ab01b61bd33557 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:12.864550 4683 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94ae53a5c8dc4fc0b6ab01b61bd33557 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94ae53a5c8dc4fc0b6ab01b61bd33557, State: Initialized, Role: FOLLOWER
I20260807 11:28:12.865111 4683 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94ae53a5c8dc4fc0b6ab01b61bd33557 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94ae53a5c8dc4fc0b6ab01b61bd33557" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 41205 } }
I20260807 11:28:12.865324 4683 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94ae53a5c8dc4fc0b6ab01b61bd33557 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:28:12.865423 4683 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94ae53a5c8dc4fc0b6ab01b61bd33557 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:28:12.865599 4683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94ae53a5c8dc4fc0b6ab01b61bd33557 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:12.868283 4683 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94ae53a5c8dc4fc0b6ab01b61bd33557 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94ae53a5c8dc4fc0b6ab01b61bd33557" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 41205 } }
I20260807 11:28:12.868750 4683 leader_election.cc:302] T 00000000000000000000000000000000 P 94ae53a5c8dc4fc0b6ab01b61bd33557 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94ae53a5c8dc4fc0b6ab01b61bd33557; no voters:
I20260807 11:28:12.869325 4683 leader_election.cc:288] T 00000000000000000000000000000000 P 94ae53a5c8dc4fc0b6ab01b61bd33557 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:28:12.869481 4688 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94ae53a5c8dc4fc0b6ab01b61bd33557 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:12.870250 4688 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94ae53a5c8dc4fc0b6ab01b61bd33557 [term 1 LEADER]: Becoming Leader. State: Replica: 94ae53a5c8dc4fc0b6ab01b61bd33557, State: Running, Role: LEADER
I20260807 11:28:12.870873 4688 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94ae53a5c8dc4fc0b6ab01b61bd33557 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94ae53a5c8dc4fc0b6ab01b61bd33557" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 41205 } }
I20260807 11:28:12.871613 4683 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94ae53a5c8dc4fc0b6ab01b61bd33557 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:28:12.873719 4690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94ae53a5c8dc4fc0b6ab01b61bd33557 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94ae53a5c8dc4fc0b6ab01b61bd33557. Latest consensus state: current_term: 1 leader_uuid: "94ae53a5c8dc4fc0b6ab01b61bd33557" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94ae53a5c8dc4fc0b6ab01b61bd33557" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 41205 } } }
I20260807 11:28:12.873821 4689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94ae53a5c8dc4fc0b6ab01b61bd33557 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94ae53a5c8dc4fc0b6ab01b61bd33557" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94ae53a5c8dc4fc0b6ab01b61bd33557" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 41205 } } }
I20260807 11:28:12.874222 4689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94ae53a5c8dc4fc0b6ab01b61bd33557 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:12.874226 4690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94ae53a5c8dc4fc0b6ab01b61bd33557 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:12.875365 4695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:28:12.880843 4695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:28:12.895290 4695 catalog_manager.cc:1347] Generated new cluster ID: 8e9ef23d696d418995d19542a1aa6f56
I20260807 11:28:12.895426 4695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:28:12.911123 4695 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:28:12.912405 4695 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:28:12.920637 4695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94ae53a5c8dc4fc0b6ab01b61bd33557: Generated new TSK 0
I20260807 11:28:12.921563 4695 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:28:12.940181 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:0
--local_ip_for_outbound_sockets=127.16.197.1
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:41205
--builtin_ntp_servers=127.16.197.20:42177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 11:28:13.301102 4707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:13.301400 4707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:13.301556 4707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:13.308939 4707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:13.309051 4707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:13.309226 4707 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:28:13.318436 4707 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:42177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:41205
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:13.319983 4707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:13.321852 4707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:13.335225 4713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:13.335225 4715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:13.335387 4712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:13.336081 4707 server_base.cc:1034] running on GCE node
I20260807 11:28:13.336640 4707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:13.337538 4707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:13.338780 4707 hybrid_clock.cc:648] HybridClock initialized: now 1786102093338725 us; error 45 us; skew 500 ppm
I20260807 11:28:13.339172 4707 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 11:28:13.341444 4707 webserver.cc:460] Webserver started at http://127.16.197.1:45031/ using document root <none> and password file <none>
I20260807 11:28:13.342134 4707 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:13.342263 4707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:13.342584 4707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:13.345520 4707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-0/data/instance:
uuid: "f7f4facd21a24eaaa4405f0e16a057c1"
format_stamp: "Formatted at 2026-08-07 11:28:13 on dist-test-slave-2w3w"
server_key: "59db01ee17c247a82c7476bd0bbbe9b0"
server_key_iv: "3164608e5ddf56038e2c1994b3a10777"
server_key_version: "encryptionkey@0"
I20260807 11:28:13.346333 4707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance:
uuid: "f7f4facd21a24eaaa4405f0e16a057c1"
format_stamp: "Formatted at 2026-08-07 11:28:13 on dist-test-slave-2w3w"
server_key: "59db01ee17c247a82c7476bd0bbbe9b0"
server_key_iv: "3164608e5ddf56038e2c1994b3a10777"
server_key_version: "encryptionkey@0"
I20260807 11:28:13.351662 4707 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260807 11:28:13.355202 4721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:13.356714 4707 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 11:28:13.356900 4707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "f7f4facd21a24eaaa4405f0e16a057c1"
format_stamp: "Formatted at 2026-08-07 11:28:13 on dist-test-slave-2w3w"
server_key: "59db01ee17c247a82c7476bd0bbbe9b0"
server_key_iv: "3164608e5ddf56038e2c1994b3a10777"
server_key_version: "encryptionkey@0"
I20260807 11:28:13.357128 4707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:13.375267 4707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:13.376154 4707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:13.376461 4707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:13.377396 4707 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:28:13.378878 4707 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:28:13.379030 4707 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:13.379122 4707 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:28:13.379185 4707 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:13.412878 4707 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:40003
I20260807 11:28:13.412915 4838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:40003 every 8 connection(s)
I20260807 11:28:13.414254 4707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:28:13.417449 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 4707
I20260807 11:28:13.417830 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance
I20260807 11:28:13.418398 17172 external_mini_cluster.cc:1428] Setting key 73f12bc43de86d82065e5c972191c39a
I20260807 11:28:13.421638 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:0
--local_ip_for_outbound_sockets=127.16.197.2
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:41205
--builtin_ntp_servers=127.16.197.20:42177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 11:28:13.428614 4841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:13.416916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:40849 (local address 127.16.197.62:41205)
0807 11:28:13.418098 (+ 1182us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:13.418105 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:13.419099 (+ 994us) server_negotiation.cc:407] Connection header received
0807 11:28:13.421179 (+ 2080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:13.421187 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:13.421280 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:13.421432 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:13.422632 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:13.423857 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:13.425384 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:13.426082 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:13.426225 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:13.427606 (+ 1381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:13.427616 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:13.427623 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:13.427649 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:13.427704 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:13.427725 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:13.427839 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:13.428033 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 11:28:13.428156 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1040,"thread_start_us":910,"threads_started":1}
I20260807 11:28:13.429414 4842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:13.417729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:41205 (local address 127.16.197.1:40849)
0807 11:28:13.418891 (+ 1162us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:13.418937 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:13.420917 (+ 1980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:13.421697 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:13.421705 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:13.421858 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:28:13.422453 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:13.422471 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:13.424010 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:13.424013 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:13.425226 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:13.425238 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:13.426280 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:13.426285 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:13.426482 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:13.427243 (+ 761us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 11:28:13.427245 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:28:13.427315 (+ 70us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 11:28:13.427393 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:28:13.427833 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:28:13.427841 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:28:13.427846 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 11:28:13.427987 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 11:28:13.428292 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":375,"threads_started":1}
I20260807 11:28:13.431025 4839 heartbeater.cc:347] Connected to a master server at 127.16.197.62:41205
I20260807 11:28:13.431375 4839 heartbeater.cc:464] Registering TS with master...
I20260807 11:28:13.432319 4839 heartbeater.cc:511] Master 127.16.197.62:41205 requested a full tablet report, sending...
I20260807 11:28:13.434602 4646 ts_manager.cc:194] Registered new tserver with Master: f7f4facd21a24eaaa4405f0e16a057c1 (127.16.197.1:40003)
I20260807 11:28:13.436110 4646 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.197.1:40849
W20260807 11:28:13.763994 4843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:13.764323 4843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:13.764492 4843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:13.771926 4843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:13.772056 4843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:13.772254 4843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:28:13.781709 4843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:42177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:41205
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:13.783115 4843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:13.785013 4843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:13.798978 4852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:13.798887 4850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:13.798931 4849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:13.799675 4843 server_base.cc:1034] running on GCE node
I20260807 11:28:13.800313 4843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:13.801270 4843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:13.802465 4843 hybrid_clock.cc:648] HybridClock initialized: now 1786102093802397 us; error 68 us; skew 500 ppm
I20260807 11:28:13.802809 4843 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 11:28:13.805189 4843 webserver.cc:460] Webserver started at http://127.16.197.2:46493/ using document root <none> and password file <none>
I20260807 11:28:13.805918 4843 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:13.806008 4843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:13.806352 4843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:13.809335 4843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-1/data/instance:
uuid: "f70fbff671ef40ec873749e4ddc32426"
format_stamp: "Formatted at 2026-08-07 11:28:13 on dist-test-slave-2w3w"
server_key: "262c13d62afc2b1021ff1ca0e3bbc22a"
server_key_iv: "f7ae3a37d6efcc9857c15d889a1258a0"
server_key_version: "encryptionkey@0"
I20260807 11:28:13.810114 4843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance:
uuid: "f70fbff671ef40ec873749e4ddc32426"
format_stamp: "Formatted at 2026-08-07 11:28:13 on dist-test-slave-2w3w"
server_key: "262c13d62afc2b1021ff1ca0e3bbc22a"
server_key_iv: "f7ae3a37d6efcc9857c15d889a1258a0"
server_key_version: "encryptionkey@0"
I20260807 11:28:13.815577 4843 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 11:28:13.819125 4858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:13.820415 4843 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 11:28:13.820595 4843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "f70fbff671ef40ec873749e4ddc32426"
format_stamp: "Formatted at 2026-08-07 11:28:13 on dist-test-slave-2w3w"
server_key: "262c13d62afc2b1021ff1ca0e3bbc22a"
server_key_iv: "f7ae3a37d6efcc9857c15d889a1258a0"
server_key_version: "encryptionkey@0"
I20260807 11:28:13.820780 4843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:13.839818 4843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:13.840690 4843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:13.840989 4843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:13.842051 4843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:28:13.843900 4843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:28:13.844007 4843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:13.844131 4843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:28:13.844213 4843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:13.877465 4843 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:35883
I20260807 11:28:13.877514 4974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:35883 every 8 connection(s)
I20260807 11:28:13.878865 4843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:28:13.883416 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 4843
I20260807 11:28:13.883618 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance
I20260807 11:28:13.884227 17172 external_mini_cluster.cc:1428] Setting key 0c0639fc00d6013a0bd5368ac991e800
I20260807 11:28:13.887571 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:0
--local_ip_for_outbound_sockets=127.16.197.3
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:41205
--builtin_ntp_servers=127.16.197.20:42177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 11:28:13.894124 4841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:13.881602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:42201 (local address 127.16.197.62:41205)
0807 11:28:13.881845 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:13.881861 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:13.883411 (+ 1550us) server_negotiation.cc:407] Connection header received
0807 11:28:13.885723 (+ 2312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:13.885729 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:13.885824 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:13.885941 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:13.887402 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:13.888257 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:13.891051 (+ 2794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:13.891470 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:13.891574 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:13.893354 (+ 1780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:13.893359 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:13.893362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:13.893381 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:13.893415 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:13.893427 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:13.893503 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:13.893779 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 11:28:13.893882 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 11:28:13.895828 4978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:13.882158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:41205 (local address 127.16.197.2:42201)
0807 11:28:13.883184 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:13.883250 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:13.885439 (+ 2189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:13.886192 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:13.886201 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:13.886298 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:28:13.887209 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:13.887240 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:13.888435 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:13.888443 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:13.890829 (+ 2386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:13.890849 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:13.891710 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:13.891719 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:13.892059 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:13.892939 (+ 880us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 11:28:13.892945 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:28:13.893000 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 11:28:13.893113 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:28:13.893529 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:28:13.893542 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:28:13.893553 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 11:28:13.893753 (+ 200us) client_negotiation.cc:240] Negotiation successful
0807 11:28:13.894250 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":283,"threads_started":1}
I20260807 11:28:13.897382 4976 heartbeater.cc:347] Connected to a master server at 127.16.197.62:41205
I20260807 11:28:13.897725 4976 heartbeater.cc:464] Registering TS with master...
I20260807 11:28:13.898754 4976 heartbeater.cc:511] Master 127.16.197.62:41205 requested a full tablet report, sending...
I20260807 11:28:13.900621 4646 ts_manager.cc:194] Registered new tserver with Master: f70fbff671ef40ec873749e4ddc32426 (127.16.197.2:35883)
I20260807 11:28:13.901966 4646 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.197.2:42201
W20260807 11:28:14.239363 4979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:14.239637 4979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:14.239724 4979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:14.246774 4979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:14.246883 4979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:14.247046 4979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:28:14.256476 4979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:42177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:41205
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:14.257722 4979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:14.259624 4979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:14.272997 4985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:14.272903 4986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:14.273327 4988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:14.274199 4979 server_base.cc:1034] running on GCE node
I20260807 11:28:14.274834 4979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:14.275902 4979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:14.277122 4979 hybrid_clock.cc:648] HybridClock initialized: now 1786102094277079 us; error 37 us; skew 500 ppm
I20260807 11:28:14.277436 4979 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 11:28:14.279664 4979 webserver.cc:460] Webserver started at http://127.16.197.3:42015/ using document root <none> and password file <none>
I20260807 11:28:14.280339 4979 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:14.280465 4979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:14.280773 4979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:14.283622 4979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-2/data/instance:
uuid: "52285729c942429498303907c71656bb"
format_stamp: "Formatted at 2026-08-07 11:28:14 on dist-test-slave-2w3w"
server_key: "b9ba7847bd09e740e421f454a1d89aa0"
server_key_iv: "1bbd4a47575cf2a617a4059d26c09d5d"
server_key_version: "encryptionkey@0"
I20260807 11:28:14.284469 4979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance:
uuid: "52285729c942429498303907c71656bb"
format_stamp: "Formatted at 2026-08-07 11:28:14 on dist-test-slave-2w3w"
server_key: "b9ba7847bd09e740e421f454a1d89aa0"
server_key_iv: "1bbd4a47575cf2a617a4059d26c09d5d"
server_key_version: "encryptionkey@0"
I20260807 11:28:14.289427 4979 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 11:28:14.292876 4994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:14.294226 4979 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 11:28:14.294402 4979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "52285729c942429498303907c71656bb"
format_stamp: "Formatted at 2026-08-07 11:28:14 on dist-test-slave-2w3w"
server_key: "b9ba7847bd09e740e421f454a1d89aa0"
server_key_iv: "1bbd4a47575cf2a617a4059d26c09d5d"
server_key_version: "encryptionkey@0"
I20260807 11:28:14.294612 4979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:14.312284 4979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:14.313130 4979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:14.313441 4979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:14.314416 4979 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:28:14.315963 4979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:28:14.316076 4979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:14.316195 4979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:28:14.316275 4979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:14.350921 4979 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:43563
I20260807 11:28:14.351037 5113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:43563 every 8 connection(s)
I20260807 11:28:14.352420 4979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:28:14.360165 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 4979
I20260807 11:28:14.360376 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance
I20260807 11:28:14.360843 17172 external_mini_cluster.cc:1428] Setting key 9390526d9723cd6ace0bde7e8bf2b08a
I20260807 11:28:14.365736 4841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:14.355166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:51363 (local address 127.16.197.62:41205)
0807 11:28:14.355393 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:14.355399 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:14.356882 (+ 1483us) server_negotiation.cc:407] Connection header received
0807 11:28:14.358692 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:14.358697 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:14.358767 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:14.358860 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:14.360091 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.360989 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.363087 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.363483 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.363605 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:14.365041 (+ 1436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:14.365047 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:14.365050 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:14.365067 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:14.365100 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:14.365112 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:14.365175 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:14.365395 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 11:28:14.365475 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 11:28:14.366714 5116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:14.355768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:41205 (local address 127.16.197.3:51363)
0807 11:28:14.356722 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:14.356766 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:14.358416 (+ 1650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:14.359123 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:14.359131 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:14.359217 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:28:14.359892 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:14.359921 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.361160 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.361165 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:14.362918 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:14.362938 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.363753 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.363763 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:14.364046 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:14.364692 (+ 646us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 11:28:14.364695 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:28:14.364761 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 11:28:14.364851 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:28:14.365189 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:28:14.365196 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:28:14.365202 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 11:28:14.365356 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 11:28:14.365661 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":289,"threads_started":1}
I20260807 11:28:14.368084 5114 heartbeater.cc:347] Connected to a master server at 127.16.197.62:41205
I20260807 11:28:14.368357 5114 heartbeater.cc:464] Registering TS with master...
I20260807 11:28:14.369043 5114 heartbeater.cc:511] Master 127.16.197.62:41205 requested a full tablet report, sending...
I20260807 11:28:14.370394 4646 ts_manager.cc:194] Registered new tserver with Master: 52285729c942429498303907c71656bb (127.16.197.3:43563)
I20260807 11:28:14.371420 4646 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.197.3:51363
I20260807 11:28:14.376154 17172 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 11:28:14.386250 4841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:14.380388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55878 (local address 127.16.197.62:41205)
0807 11:28:14.380549 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:14.380553 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:14.381045 (+ 492us) server_negotiation.cc:407] Connection header received
0807 11:28:14.381169 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:14.381173 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:14.381237 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:14.381322 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:14.381708 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.382424 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.383969 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.384370 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.384479 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:14.385498 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:14.385502 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:14.385506 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:14.385525 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:14.385559 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:14.385571 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:14.385685 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:14.385837 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 11:28:14.385927 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 11:28:14.390062 4646 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55878:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 11:28:14.396981 4646 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:28:14.439750 4839 heartbeater.cc:503] Master 127.16.197.62:41205 was elected leader, sending a full tablet report...
I20260807 11:28:14.441524 5127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:14.409413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:40003 (local address 127.0.0.1:54474)
0807 11:28:14.431815 (+ 22402us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:14.431841 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:14.432002 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:14.432975 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:14.432985 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:14.433048 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:14.433756 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:14.433770 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.436139 (+ 2369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.436144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:14.438768 (+ 2624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:14.438779 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.440644 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.440649 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:14.440820 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:14.440861 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 11:28:14.440958 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 11:28:14.441081 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22217,"mutex_wait_us":52,"spinlock_wait_cycles":16128,"thread_start_us":22039,"threads_started":1}
I20260807 11:28:14.441776 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:14.409520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51720 (local address 127.16.197.3:43563)
0807 11:28:14.432457 (+ 22937us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:14.432465 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:14.432498 (+ 33us) server_negotiation.cc:407] Connection header received
0807 11:28:14.432594 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:14.432604 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:14.432838 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:14.433948 (+ 1110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:14.434246 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.435843 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.438288 (+ 2445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.439766 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.439912 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:14.440203 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:14.441206 (+ 1003us) server_negotiation.cc:299] Negotiation successful
0807 11:28:14.441382 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22776,"thread_start_us":22629,"threads_started":1}
I20260807 11:28:14.442070 5126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:14.409773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:35883 (local address 127.0.0.1:39078)
0807 11:28:14.432798 (+ 23025us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:14.432814 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:14.433043 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:14.433891 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:14.433896 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:14.433936 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:14.434198 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:14.434205 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.435700 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.435708 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:14.438280 (+ 2572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:14.438295 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.441249 (+ 2954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.441265 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:14.441429 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:14.441466 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 11:28:14.441566 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 11:28:14.441692 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22903,"mutex_wait_us":3,"spinlock_wait_cycles":4864,"thread_start_us":20657,"threads_started":1}
I20260807 11:28:14.442262 5129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:14.409174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:43563 (local address 127.0.0.1:51720)
0807 11:28:14.430935 (+ 21761us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:14.431019 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:14.431416 (+ 397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:14.433037 (+ 1621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:14.433041 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:14.433061 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:14.433757 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:14.433770 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.436036 (+ 2266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.436044 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:14.438048 (+ 2004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:14.438061 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.440346 (+ 2285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.440353 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:14.440596 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:14.440659 (+ 63us) client_negotiation.cc:769] Sending connection context
0807 11:28:14.440816 (+ 157us) client_negotiation.cc:240] Negotiation successful
0807 11:28:14.441007 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21432,"mutex_wait_us":87,"thread_start_us":23363,"threads_started":1}
I20260807 11:28:14.442586 5131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:14.410118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39078 (local address 127.16.197.2:35883)
0807 11:28:14.430014 (+ 19896us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:14.430027 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:14.432914 (+ 2887us) server_negotiation.cc:407] Connection header received
0807 11:28:14.433254 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:14.433262 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:14.433452 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:14.433610 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:14.434443 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.435566 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.438475 (+ 2909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.440787 (+ 2312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.440909 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:14.441113 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:14.442041 (+ 928us) server_negotiation.cc:299] Negotiation successful
0807 11:28:14.442235 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19661,"thread_start_us":19914,"threads_started":1}
I20260807 11:28:14.443025 5130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:14.409909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54474 (local address 127.16.197.1:40003)
0807 11:28:14.432052 (+ 22143us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:14.432063 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:14.432108 (+ 45us) server_negotiation.cc:407] Connection header received
0807 11:28:14.432215 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:14.432223 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:14.432488 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:14.432683 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:14.433974 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.435772 (+ 1798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.438980 (+ 3208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.440123 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.440255 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:14.440509 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:14.442477 (+ 1968us) server_negotiation.cc:299] Negotiation successful
0807 11:28:14.442652 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21963,"thread_start_us":21868,"threads_started":1}
I20260807 11:28:14.446010 4907 tablet_service.cc:1467] Processing CreateTablet for tablet 7fd44b82b6c94b90b94e1250f2653959 (DEFAULT_TABLE table=test-table [id=ce6b6889b62b4888b0d991ac989851ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:28:14.446285 5044 tablet_service.cc:1467] Processing CreateTablet for tablet 7fd44b82b6c94b90b94e1250f2653959 (DEFAULT_TABLE table=test-table [id=ce6b6889b62b4888b0d991ac989851ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:28:14.446853 4772 tablet_service.cc:1467] Processing CreateTablet for tablet 7fd44b82b6c94b90b94e1250f2653959 (DEFAULT_TABLE table=test-table [id=ce6b6889b62b4888b0d991ac989851ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:28:14.447603 4907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fd44b82b6c94b90b94e1250f2653959. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:14.447752 5044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fd44b82b6c94b90b94e1250f2653959. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:14.448110 4772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fd44b82b6c94b90b94e1250f2653959. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:14.459220 5132 tablet_bootstrap.cc:492] T 7fd44b82b6c94b90b94e1250f2653959 P f70fbff671ef40ec873749e4ddc32426: Bootstrap starting.
I20260807 11:28:14.462399 5135 tablet_bootstrap.cc:492] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb: Bootstrap starting.
I20260807 11:28:14.462852 5132 tablet_bootstrap.cc:654] T 7fd44b82b6c94b90b94e1250f2653959 P f70fbff671ef40ec873749e4ddc32426: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:14.464294 5132 log.cc:824] T 7fd44b82b6c94b90b94e1250f2653959 P f70fbff671ef40ec873749e4ddc32426: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:14.465183 5134 tablet_bootstrap.cc:492] T 7fd44b82b6c94b90b94e1250f2653959 P f7f4facd21a24eaaa4405f0e16a057c1: Bootstrap starting.
I20260807 11:28:14.466048 5135 tablet_bootstrap.cc:654] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:14.467521 5135 log.cc:824] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:14.468447 5134 tablet_bootstrap.cc:654] T 7fd44b82b6c94b90b94e1250f2653959 P f7f4facd21a24eaaa4405f0e16a057c1: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:14.469621 5132 tablet_bootstrap.cc:492] T 7fd44b82b6c94b90b94e1250f2653959 P f70fbff671ef40ec873749e4ddc32426: No bootstrap required, opened a new log
I20260807 11:28:14.469650 5134 log.cc:824] T 7fd44b82b6c94b90b94e1250f2653959 P f7f4facd21a24eaaa4405f0e16a057c1: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:14.469930 5132 ts_tablet_manager.cc:1397] T 7fd44b82b6c94b90b94e1250f2653959 P f70fbff671ef40ec873749e4ddc32426: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260807 11:28:14.470129 5135 tablet_bootstrap.cc:492] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb: No bootstrap required, opened a new log
I20260807 11:28:14.470453 5135 ts_tablet_manager.cc:1397] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260807 11:28:14.473016 5134 tablet_bootstrap.cc:492] T 7fd44b82b6c94b90b94e1250f2653959 P f7f4facd21a24eaaa4405f0e16a057c1: No bootstrap required, opened a new log
I20260807 11:28:14.473274 5134 ts_tablet_manager.cc:1397] T 7fd44b82b6c94b90b94e1250f2653959 P f7f4facd21a24eaaa4405f0e16a057c1: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260807 11:28:14.478830 5135 raft_consensus.cc:348] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52285729c942429498303907c71656bb" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 43563 } } peers { permanent_uuid: "f7f4facd21a24eaaa4405f0e16a057c1" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 40003 } } peers { permanent_uuid: "f70fbff671ef40ec873749e4ddc32426" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35883 } }
I20260807 11:28:14.479277 5135 raft_consensus.cc:374] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:14.479393 5135 raft_consensus.cc:729] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52285729c942429498303907c71656bb, State: Initialized, Role: FOLLOWER
I20260807 11:28:14.480049 5135 consensus_queue.cc:260] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52285729c942429498303907c71656bb" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 43563 } } peers { permanent_uuid: "f7f4facd21a24eaaa4405f0e16a057c1" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 40003 } } peers { permanent_uuid: "f70fbff671ef40ec873749e4ddc32426" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35883 } }
I20260807 11:28:14.481158 5114 heartbeater.cc:503] Master 127.16.197.62:41205 was elected leader, sending a full tablet report...
I20260807 11:28:14.480903 5132 raft_consensus.cc:348] T 7fd44b82b6c94b90b94e1250f2653959 P f70fbff671ef40ec873749e4ddc32426 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52285729c942429498303907c71656bb" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 43563 } } peers { permanent_uuid: "f7f4facd21a24eaaa4405f0e16a057c1" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 40003 } } peers { permanent_uuid: "f70fbff671ef40ec873749e4ddc32426" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35883 } }
I20260807 11:28:14.481307 5132 raft_consensus.cc:374] T 7fd44b82b6c94b90b94e1250f2653959 P f70fbff671ef40ec873749e4ddc32426 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:14.481400 5132 raft_consensus.cc:729] T 7fd44b82b6c94b90b94e1250f2653959 P f70fbff671ef40ec873749e4ddc32426 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f70fbff671ef40ec873749e4ddc32426, State: Initialized, Role: FOLLOWER
I20260807 11:28:14.481889 5135 ts_tablet_manager.cc:1428] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb: Time spent starting tablet: real 0.011s user 0.006s sys 0.004s
I20260807 11:28:14.482088 5132 consensus_queue.cc:260] T 7fd44b82b6c94b90b94e1250f2653959 P f70fbff671ef40ec873749e4ddc32426 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52285729c942429498303907c71656bb" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 43563 } } peers { permanent_uuid: "f7f4facd21a24eaaa4405f0e16a057c1" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 40003 } } peers { permanent_uuid: "f70fbff671ef40ec873749e4ddc32426" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35883 } }
I20260807 11:28:14.483779 5132 ts_tablet_manager.cc:1428] T 7fd44b82b6c94b90b94e1250f2653959 P f70fbff671ef40ec873749e4ddc32426: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260807 11:28:14.483896 5134 raft_consensus.cc:348] T 7fd44b82b6c94b90b94e1250f2653959 P f7f4facd21a24eaaa4405f0e16a057c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52285729c942429498303907c71656bb" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 43563 } } peers { permanent_uuid: "f7f4facd21a24eaaa4405f0e16a057c1" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 40003 } } peers { permanent_uuid: "f70fbff671ef40ec873749e4ddc32426" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35883 } }
I20260807 11:28:14.484230 4976 heartbeater.cc:503] Master 127.16.197.62:41205 was elected leader, sending a full tablet report...
I20260807 11:28:14.484385 5134 raft_consensus.cc:374] T 7fd44b82b6c94b90b94e1250f2653959 P f7f4facd21a24eaaa4405f0e16a057c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:14.484483 5134 raft_consensus.cc:729] T 7fd44b82b6c94b90b94e1250f2653959 P f7f4facd21a24eaaa4405f0e16a057c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7f4facd21a24eaaa4405f0e16a057c1, State: Initialized, Role: FOLLOWER
I20260807 11:28:14.485152 5134 consensus_queue.cc:260] T 7fd44b82b6c94b90b94e1250f2653959 P f7f4facd21a24eaaa4405f0e16a057c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52285729c942429498303907c71656bb" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 43563 } } peers { permanent_uuid: "f7f4facd21a24eaaa4405f0e16a057c1" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 40003 } } peers { permanent_uuid: "f70fbff671ef40ec873749e4ddc32426" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35883 } }
I20260807 11:28:14.487318 5134 ts_tablet_manager.cc:1428] T 7fd44b82b6c94b90b94e1250f2653959 P f7f4facd21a24eaaa4405f0e16a057c1: Time spent starting tablet: real 0.014s user 0.008s sys 0.005s
W20260807 11:28:14.604269 5115 tablet.cc:2369] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 11:28:14.630988 4977 tablet.cc:2369] T 7fd44b82b6c94b90b94e1250f2653959 P f70fbff671ef40ec873749e4ddc32426: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 11:28:14.666711 4840 tablet.cc:2369] T 7fd44b82b6c94b90b94e1250f2653959 P f7f4facd21a24eaaa4405f0e16a057c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:28:14.679883 5139 raft_consensus.cc:482] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:28:14.680192 5139 raft_consensus.cc:504] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52285729c942429498303907c71656bb" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 43563 } } peers { permanent_uuid: "f7f4facd21a24eaaa4405f0e16a057c1" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 40003 } } peers { permanent_uuid: "f70fbff671ef40ec873749e4ddc32426" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35883 } }
I20260807 11:28:14.681903 5139 leader_election.cc:288] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7f4facd21a24eaaa4405f0e16a057c1 (127.16.197.1:40003), f70fbff671ef40ec873749e4ddc32426 (127.16.197.2:35883)
I20260807 11:28:14.687677 5131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:14.682598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:33075 (local address 127.16.197.2:35883)
0807 11:28:14.682807 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:14.682812 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:14.682827 (+ 15us) server_negotiation.cc:407] Connection header received
0807 11:28:14.682886 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:14.682889 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:14.682995 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:14.683111 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:14.683595 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.684304 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.686039 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.686779 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.686952 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:14.687046 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:14.687297 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 11:28:14.687418 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 11:28:14.687773 5116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:14.682197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:35883 (local address 127.16.197.3:33075)
0807 11:28:14.682448 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:14.682470 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:14.682604 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:14.683155 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:14.683160 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:14.683188 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:14.683422 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:14.683431 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.684432 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.684440 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:14.685871 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:14.685884 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.686928 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.686936 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:14.687128 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:14.687153 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 11:28:14.687256 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 11:28:14.687390 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":37,"spinlock_wait_cycles":30080}
I20260807 11:28:14.688225 5130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:14.682235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:57243 (local address 127.16.197.1:40003)
0807 11:28:14.682434 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:14.682441 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:14.683241 (+ 800us) server_negotiation.cc:407] Connection header received
0807 11:28:14.683470 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:14.683474 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:14.683532 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:14.683630 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:14.684160 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.684834 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.686509 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.687331 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.687512 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:14.687578 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:14.687895 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 11:28:14.688028 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 11:28:14.688799 5142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:14.682275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:40003 (local address 127.16.197.3:57243)
0807 11:28:14.683126 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:14.683143 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:14.683305 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:14.683643 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:14.683648 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:14.683676 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:14.683995 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:14.684003 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.684985 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.684991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:14.686325 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:14.686337 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.687514 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.687520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:14.687682 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:14.687707 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 11:28:14.688465 (+ 758us) client_negotiation.cc:240] Negotiation successful
0807 11:28:14.688565 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"mutex_wait_us":1,"thread_start_us":480,"threads_started":1}
I20260807 11:28:14.688725 4929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fd44b82b6c94b90b94e1250f2653959" candidate_uuid: "52285729c942429498303907c71656bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f70fbff671ef40ec873749e4ddc32426" is_pre_election: true
I20260807 11:28:14.689253 4929 raft_consensus.cc:2454] T 7fd44b82b6c94b90b94e1250f2653959 P f70fbff671ef40ec873749e4ddc32426 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52285729c942429498303907c71656bb in term 0.
I20260807 11:28:14.689693 4794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fd44b82b6c94b90b94e1250f2653959" candidate_uuid: "52285729c942429498303907c71656bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7f4facd21a24eaaa4405f0e16a057c1" is_pre_election: true
I20260807 11:28:14.689875 4995 leader_election.cc:302] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52285729c942429498303907c71656bb, f70fbff671ef40ec873749e4ddc32426; no voters:
I20260807 11:28:14.690163 4794 raft_consensus.cc:2454] T 7fd44b82b6c94b90b94e1250f2653959 P f7f4facd21a24eaaa4405f0e16a057c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52285729c942429498303907c71656bb in term 0.
I20260807 11:28:14.690264 5139 raft_consensus.cc:2781] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:28:14.690411 5139 raft_consensus.cc:482] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:28:14.690511 5139 raft_consensus.cc:3037] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:14.692850 5139 raft_consensus.cc:504] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52285729c942429498303907c71656bb" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 43563 } } peers { permanent_uuid: "f7f4facd21a24eaaa4405f0e16a057c1" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 40003 } } peers { permanent_uuid: "f70fbff671ef40ec873749e4ddc32426" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35883 } }
I20260807 11:28:14.693485 5139 leader_election.cc:288] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb [CANDIDATE]: Term 1 election: Requested vote from peers f7f4facd21a24eaaa4405f0e16a057c1 (127.16.197.1:40003), f70fbff671ef40ec873749e4ddc32426 (127.16.197.2:35883)
I20260807 11:28:14.693864 4794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fd44b82b6c94b90b94e1250f2653959" candidate_uuid: "52285729c942429498303907c71656bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7f4facd21a24eaaa4405f0e16a057c1"
I20260807 11:28:14.693986 4929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fd44b82b6c94b90b94e1250f2653959" candidate_uuid: "52285729c942429498303907c71656bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f70fbff671ef40ec873749e4ddc32426"
I20260807 11:28:14.694089 4794 raft_consensus.cc:3037] T 7fd44b82b6c94b90b94e1250f2653959 P f7f4facd21a24eaaa4405f0e16a057c1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:14.694211 4929 raft_consensus.cc:3037] T 7fd44b82b6c94b90b94e1250f2653959 P f70fbff671ef40ec873749e4ddc32426 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:14.697479 4929 raft_consensus.cc:2454] T 7fd44b82b6c94b90b94e1250f2653959 P f70fbff671ef40ec873749e4ddc32426 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52285729c942429498303907c71656bb in term 1.
I20260807 11:28:14.697476 4794 raft_consensus.cc:2454] T 7fd44b82b6c94b90b94e1250f2653959 P f7f4facd21a24eaaa4405f0e16a057c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52285729c942429498303907c71656bb in term 1.
I20260807 11:28:14.697926 4997 leader_election.cc:302] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52285729c942429498303907c71656bb, f7f4facd21a24eaaa4405f0e16a057c1; no voters:
I20260807 11:28:14.698220 5139 raft_consensus.cc:2781] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:14.698632 5139 raft_consensus.cc:686] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb [term 1 LEADER]: Becoming Leader. State: Replica: 52285729c942429498303907c71656bb, State: Running, Role: LEADER
I20260807 11:28:14.699226 5139 consensus_queue.cc:237] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52285729c942429498303907c71656bb" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 43563 } } peers { permanent_uuid: "f7f4facd21a24eaaa4405f0e16a057c1" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 40003 } } peers { permanent_uuid: "f70fbff671ef40ec873749e4ddc32426" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35883 } }
I20260807 11:28:14.703565 4644 catalog_manager.cc:5495] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 52285729c942429498303907c71656bb (127.16.197.3). New cstate: current_term: 1 leader_uuid: "52285729c942429498303907c71656bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52285729c942429498303907c71656bb" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 43563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7f4facd21a24eaaa4405f0e16a057c1" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 40003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f70fbff671ef40ec873749e4ddc32426" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35883 } health_report { overall_health: UNKNOWN } } }
I20260807 11:28:14.742543 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:14.737136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51728 (local address 127.16.197.3:43563)
0807 11:28:14.737345 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:14.737350 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:14.737370 (+ 20us) server_negotiation.cc:407] Connection header received
0807 11:28:14.737573 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:14.737577 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:14.737661 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:14.737803 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:28:14.738219 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.738943 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.740323 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.740733 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.740858 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:14.741460 (+ 602us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:28:14.741917 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:28:14.742089 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:14.742291 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 11:28:14.742338 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 11:28:14.750229 4929 raft_consensus.cc:1261] T 7fd44b82b6c94b90b94e1250f2653959 P f70fbff671ef40ec873749e4ddc32426 [term 1 FOLLOWER]: Refusing update from remote peer 52285729c942429498303907c71656bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:28:14.750541 4794 raft_consensus.cc:1261] T 7fd44b82b6c94b90b94e1250f2653959 P f7f4facd21a24eaaa4405f0e16a057c1 [term 1 FOLLOWER]: Refusing update from remote peer 52285729c942429498303907c71656bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:28:14.751246 5143 consensus_queue.cc:1035] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f70fbff671ef40ec873749e4ddc32426" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 35883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:28:14.751593 5139 consensus_queue.cc:1035] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7f4facd21a24eaaa4405f0e16a057c1" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 40003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:28:14.780256 4646 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:28:14.780556 4646 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:28:14.784273 4646 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 94ae53a5c8dc4fc0b6ab01b61bd33557: Sending DeleteTablet for 3 replicas of tablet 7fd44b82b6c94b90b94e1250f2653959
I20260807 11:28:14.785419 5044 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fd44b82b6c94b90b94e1250f2653959 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:28:14 UTC) from {username='slave'} at 127.0.0.1:51720
I20260807 11:28:14.785459 4772 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fd44b82b6c94b90b94e1250f2653959 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:28:14 UTC) from {username='slave'} at 127.0.0.1:54474
I20260807 11:28:14.785565 4907 tablet_service.cc:1514] Processing DeleteTablet for tablet 7fd44b82b6c94b90b94e1250f2653959 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:28:14 UTC) from {username='slave'} at 127.0.0.1:39078
I20260807 11:28:14.790828 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 4707
I20260807 11:28:14.794909 5158 tablet_replica.cc:330] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb: stopping tablet replica
I20260807 11:28:14.795480 5158 raft_consensus.cc:2229] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb [term 1 LEADER]: Raft consensus shutting down.
I20260807 11:28:14.796366 5158 raft_consensus.cc:2258] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:28:14.797818 5160 tablet_replica.cc:330] T 7fd44b82b6c94b90b94e1250f2653959 P f70fbff671ef40ec873749e4ddc32426: stopping tablet replica
I20260807 11:28:14.798235 5160 raft_consensus.cc:2229] T 7fd44b82b6c94b90b94e1250f2653959 P f70fbff671ef40ec873749e4ddc32426 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:28:14.798727 5160 raft_consensus.cc:2258] T 7fd44b82b6c94b90b94e1250f2653959 P f70fbff671ef40ec873749e4ddc32426 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:28:14.803416 5160 ts_tablet_manager.cc:1905] T 7fd44b82b6c94b90b94e1250f2653959 P f70fbff671ef40ec873749e4ddc32426: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:28:14.803948 5130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:14.791648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54490 (local address 127.16.197.1:40003)
0807 11:28:14.791858 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:14.791868 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:14.794312 (+ 2444us) server_negotiation.cc:407] Connection header received
0807 11:28:14.794911 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:14.794934 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:14.795106 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:14.795200 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:14.796031 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.797514 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.799867 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.800532 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.800668 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:14.802170 (+ 1502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:14.802181 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:14.802197 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:14.802931 (+ 734us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:14.803022 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:14.803047 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:14.803137 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:14.803433 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 11:28:14.803560 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 11:28:14.804780 5158 ts_tablet_manager.cc:1905] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:28:14.805352 4834 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:14.805395 5159 tablet_replica.cc:330] T 7fd44b82b6c94b90b94e1250f2653959 P f7f4facd21a24eaaa4405f0e16a057c1: stopping tablet replica
I20260807 11:28:14.811823 5158 ts_tablet_manager.cc:1918] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:28:14.812002 5158 log.cc:1196] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-2/wal/wals/7fd44b82b6c94b90b94e1250f2653959
I20260807 11:28:14.812469 5158 ts_tablet_manager.cc:1939] T 7fd44b82b6c94b90b94e1250f2653959 P 52285729c942429498303907c71656bb: Deleting consensus metadata
I20260807 11:28:14.813998 4629 catalog_manager.cc:4841] TS 52285729c942429498303907c71656bb (127.16.197.3:43563): tablet 7fd44b82b6c94b90b94e1250f2653959 (table test-table [id=ce6b6889b62b4888b0d991ac989851ed]) successfully deleted
I20260807 11:28:14.816008 5160 ts_tablet_manager.cc:1918] T 7fd44b82b6c94b90b94e1250f2653959 P f70fbff671ef40ec873749e4ddc32426: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:28:14.816191 5160 log.cc:1196] T 7fd44b82b6c94b90b94e1250f2653959 P f70fbff671ef40ec873749e4ddc32426: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786101765888975-17172-0/minicluster-data/ts-1/wal/wals/7fd44b82b6c94b90b94e1250f2653959
I20260807 11:28:14.816759 5160 ts_tablet_manager.cc:1939] T 7fd44b82b6c94b90b94e1250f2653959 P f70fbff671ef40ec873749e4ddc32426: Deleting consensus metadata
I20260807 11:28:14.818698 4628 catalog_manager.cc:4841] TS f70fbff671ef40ec873749e4ddc32426 (127.16.197.2:35883): tablet 7fd44b82b6c94b90b94e1250f2653959 (table test-table [id=ce6b6889b62b4888b0d991ac989851ed]) successfully deleted
I20260807 11:28:14.805722 5159 raft_consensus.cc:2229] T 7fd44b82b6c94b90b94e1250f2653959 P f7f4facd21a24eaaa4405f0e16a057c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:28:14.898128 5159 raft_consensus.cc:2258] T 7fd44b82b6c94b90b94e1250f2653959 P f7f4facd21a24eaaa4405f0e16a057c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:28:14.898367 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 4707
W20260807 11:28:14.918696 4630 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.197.1:40003 (error 108)
I20260807 11:28:14.920200 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 4843
I20260807 11:28:14.920570 5127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:14.920133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:40003 (local address 127.0.0.1:54494)
0807 11:28:14.920313 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:14.920432 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 11:28:14.921051 4630 catalog_manager.cc:4584] TS f7f4facd21a24eaaa4405f0e16a057c1 (127.16.197.1:40003): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7fd44b82b6c94b90b94e1250f2653959: Network error: Client connection negotiation failed: client connection to 127.16.197.1:40003: connect: Connection refused (error 111)
I20260807 11:28:14.927340 5131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:14.921047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39086 (local address 127.16.197.2:35883)
0807 11:28:14.921221 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:14.921228 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:14.921373 (+ 145us) server_negotiation.cc:407] Connection header received
0807 11:28:14.921489 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:14.921493 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:14.921592 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:14.921709 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:14.922119 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.922880 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.924295 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:14.924697 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:14.924827 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:14.925955 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:14.925964 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:14.925977 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:14.926560 (+ 583us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:14.926606 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:14.926625 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:14.926747 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:14.926876 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 11:28:14.927016 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 11:28:14.928033 4970 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:14.974788 5127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:14.974203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:40003 (local address 127.0.0.1:54502)
0807 11:28:14.974472 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:14.974598 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 11:28:14.976068 5127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:14.975663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:40003 (local address 127.0.0.1:54512)
0807 11:28:14.975833 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:14.975930 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 11:28:15.023974 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 4843
I20260807 11:28:15.036777 5127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:15.036125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:40003 (local address 127.0.0.1:54520)
0807 11:28:15.036416 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:15.036537 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260807 11:28:15.038066 5127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:15.037605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:40003 (local address 127.0.0.1:54524)
0807 11:28:15.037782 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:15.037899 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 11:28:15.044971 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 4979
I20260807 11:28:15.054430 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:15.046214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51740 (local address 127.16.197.3:43563)
0807 11:28:15.046413 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:15.046419 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:15.046442 (+ 23us) server_negotiation.cc:407] Connection header received
0807 11:28:15.046645 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:15.046651 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:15.046753 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:15.046871 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:15.047503 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:15.048860 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:15.050746 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:15.051444 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:15.051564 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:15.052927 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:15.052936 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:15.052950 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:15.053642 (+ 692us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:15.053689 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:15.053708 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:15.053830 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:15.053964 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 11:28:15.054060 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 11:28:15.055142 5109 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:15.119082 5127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:15.118437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:40003 (local address 127.0.0.1:54536)
0807 11:28:15.118694 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:15.118792 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260807 11:28:15.120208 5127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:15.119837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:40003 (local address 127.0.0.1:54552)
0807 11:28:15.120032 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:15.120103 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:40003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260807 11:28:15.178315 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 4979
I20260807 11:28:15.199662 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 4611
I20260807 11:28:15.200860 4677 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:15.311786 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 4611
2026-08-07T11:28:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3021 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:28:15 dist-test-slave-2w3w krb5kdc[5169](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:28:15 dist-test-slave-2w3w krb5kdc[5169](info): set up 2 sockets
Aug 07 11:28:15 dist-test-slave-2w3w krb5kdc[5169](info): commencing operation
krb5kdc: starting...
W20260807 11:28:16.465307 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:28:16 dist-test-slave-2w3w krb5kdc[5169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:28:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:28:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:28:16.754521 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:41791
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:40925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:41791
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:17.101672 5182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:17.101971 5182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:17.102098 5182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:17.109468 5182 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:28:17.109597 5182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:17.109647 5182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:17.109753 5182 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:28:17.109826 5182 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:28:17.119845 5182 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:40925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:41791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:41791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:17.121246 5182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:17.122606 5182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:17.134691 5187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:17.134747 5190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:17.134773 5188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:17.135766 5182 server_base.cc:1034] running on GCE node
I20260807 11:28:17.136576 5182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:17.138110 5182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:17.139364 5182 hybrid_clock.cc:648] HybridClock initialized: now 1786102097139296 us; error 51 us; skew 500 ppm
I20260807 11:28:17.139724 5182 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:17 dist-test-slave-2w3w krb5kdc[5169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102097, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:17.148124 5182 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:28:17.149778 5182 webserver.cc:460] Webserver started at http://127.16.197.62:36073/ using document root <none> and password file <none>
I20260807 11:28:17.150485 5182 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:17.150595 5182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:17.150880 5182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:17.154150 5182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "3c8df210e8594ddca88be898b00eb18a"
format_stamp: "Formatted at 2026-08-07 11:28:17 on dist-test-slave-2w3w"
server_key: "6bdbe77660c1def76738babd4243650e"
server_key_iv: "b1a924f1633187ad49f3b5b359270770"
server_key_version: "encryptionkey@0"
I20260807 11:28:17.154918 5182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "3c8df210e8594ddca88be898b00eb18a"
format_stamp: "Formatted at 2026-08-07 11:28:17 on dist-test-slave-2w3w"
server_key: "6bdbe77660c1def76738babd4243650e"
server_key_iv: "b1a924f1633187ad49f3b5b359270770"
server_key_version: "encryptionkey@0"
I20260807 11:28:17.160396 5182 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 11:28:17.164100 5197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:17.165457 5182 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 11:28:17.165652 5182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "3c8df210e8594ddca88be898b00eb18a"
format_stamp: "Formatted at 2026-08-07 11:28:17 on dist-test-slave-2w3w"
server_key: "6bdbe77660c1def76738babd4243650e"
server_key_iv: "b1a924f1633187ad49f3b5b359270770"
server_key_version: "encryptionkey@0"
I20260807 11:28:17.165868 5182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:17.188079 5182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:17.192415 5182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:17.192723 5182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:17.211730 5182 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:41791
I20260807 11:28:17.211773 5250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:41791 every 8 connection(s)
I20260807 11:28:17.213195 5182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:28:17.216965 5251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:17.222783 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 5182
I20260807 11:28:17.222971 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:28:17.223440 17172 external_mini_cluster.cc:1428] Setting key 41f1cd5c4aebf4dd4d12909768694f24
I20260807 11:28:17.228232 5251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c8df210e8594ddca88be898b00eb18a: Bootstrap starting.
I20260807 11:28:17.232087 5251 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c8df210e8594ddca88be898b00eb18a: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:17.233415 5251 log.cc:824] T 00000000000000000000000000000000 P 3c8df210e8594ddca88be898b00eb18a: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:17.236965 5251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c8df210e8594ddca88be898b00eb18a: No bootstrap required, opened a new log
Aug 07 11:28:17 dist-test-slave-2w3w krb5kdc[5169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:28:17.248047 5251 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c8df210e8594ddca88be898b00eb18a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c8df210e8594ddca88be898b00eb18a" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 41791 } }
I20260807 11:28:17.248551 5251 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c8df210e8594ddca88be898b00eb18a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:17.248672 5251 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c8df210e8594ddca88be898b00eb18a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c8df210e8594ddca88be898b00eb18a, State: Initialized, Role: FOLLOWER
I20260807 11:28:17.249176 5254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:17.225367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42190 (local address 127.16.197.62:41791)
0807 11:28:17.226235 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:17.226247 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:17.226290 (+ 43us) server_negotiation.cc:407] Connection header received
0807 11:28:17.228076 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:17.228100 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:17.228528 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:17.228862 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:17.230845 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:17.232230 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:17.234132 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:17.235199 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:17.235328 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:17.240210 (+ 4882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:17.240220 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:17.240241 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:17.240295 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:17.244462 (+ 4167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:17.245187 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:17.245202 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:17.245211 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:17.245449 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:17.245807 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:17.245812 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:17.245816 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:17.246587 (+ 771us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:17.246921 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:17.247308 (+ 387us) server_negotiation.cc:299] Negotiation successful
0807 11:28:17.247765 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":429,"threads_started":1}
I20260807 11:28:17.249367 5251 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c8df210e8594ddca88be898b00eb18a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c8df210e8594ddca88be898b00eb18a" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 41791 } }
I20260807 11:28:17.249670 5251 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c8df210e8594ddca88be898b00eb18a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:28:17.249814 5251 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c8df210e8594ddca88be898b00eb18a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:28:17.250010 5251 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c8df210e8594ddca88be898b00eb18a [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:17.252462 5251 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c8df210e8594ddca88be898b00eb18a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c8df210e8594ddca88be898b00eb18a" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 41791 } }
I20260807 11:28:17.252955 5251 leader_election.cc:302] T 00000000000000000000000000000000 P 3c8df210e8594ddca88be898b00eb18a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c8df210e8594ddca88be898b00eb18a; no voters:
I20260807 11:28:17.253561 5251 leader_election.cc:288] T 00000000000000000000000000000000 P 3c8df210e8594ddca88be898b00eb18a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:28:17.253707 5256 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c8df210e8594ddca88be898b00eb18a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:17.254469 5256 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c8df210e8594ddca88be898b00eb18a [term 1 LEADER]: Becoming Leader. State: Replica: 3c8df210e8594ddca88be898b00eb18a, State: Running, Role: LEADER
I20260807 11:28:17.255057 5256 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c8df210e8594ddca88be898b00eb18a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c8df210e8594ddca88be898b00eb18a" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 41791 } }
I20260807 11:28:17.255767 5251 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c8df210e8594ddca88be898b00eb18a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:28:17.258666 5258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c8df210e8594ddca88be898b00eb18a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c8df210e8594ddca88be898b00eb18a. Latest consensus state: current_term: 1 leader_uuid: "3c8df210e8594ddca88be898b00eb18a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c8df210e8594ddca88be898b00eb18a" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 41791 } } }
I20260807 11:28:17.259253 5257 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c8df210e8594ddca88be898b00eb18a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c8df210e8594ddca88be898b00eb18a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c8df210e8594ddca88be898b00eb18a" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 41791 } } }
I20260807 11:28:17.259685 5258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c8df210e8594ddca88be898b00eb18a [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:17.259820 5257 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c8df210e8594ddca88be898b00eb18a [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:17.264012 5265 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:28:17.269199 5265 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:28:17.278191 5265 catalog_manager.cc:1347] Generated new cluster ID: ce086c4a201c46a9b79249d3d268ec7f
I20260807 11:28:17.278318 5265 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:28:17.303632 5265 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:28:17.305071 5265 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:28:17.314147 5265 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c8df210e8594ddca88be898b00eb18a: Generated new TSK 0
I20260807 11:28:17.315090 5265 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.197.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
Entry for principal kudu/127.16.197.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab.
I20260807 11:28:17.386582 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:0
--local_ip_for_outbound_sockets=127.16.197.1
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:41791
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:40925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:17.735363 5278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:17.735656 5278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:17.735837 5278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:17.743359 5278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:17.743486 5278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:17.743669 5278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:28:17.753624 5278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:40925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.1.keytab
--principal=kudu/127.16.197.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:41791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:17.755031 5278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:17.756476 5278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:17.770392 5283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:17.770386 5284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:17.770707 5286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:17.771615 5278 server_base.cc:1034] running on GCE node
I20260807 11:28:17.772285 5278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:17.773295 5278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:17.774497 5278 hybrid_clock.cc:648] HybridClock initialized: now 1786102097774452 us; error 34 us; skew 500 ppm
I20260807 11:28:17.774830 5278 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:17 dist-test-slave-2w3w krb5kdc[5169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102097, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:17.777938 5278 init.cc:377] Logged in from keytab as kudu/127.16.197.1@KRBTEST.COM (short username kudu)
I20260807 11:28:17.779527 5278 webserver.cc:460] Webserver started at http://127.16.197.1:39115/ using document root <none> and password file <none>
I20260807 11:28:17.780227 5278 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:17.780375 5278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:17.780684 5278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:17.783711 5278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-0/data/instance:
uuid: "2e060065878846fb8937520ba76803bf"
format_stamp: "Formatted at 2026-08-07 11:28:17 on dist-test-slave-2w3w"
server_key: "050974194f600ece444a2d786061f2bc"
server_key_iv: "5ad8532b8db10f9029a4b424bb462292"
server_key_version: "encryptionkey@0"
I20260807 11:28:17.784502 5278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance:
uuid: "2e060065878846fb8937520ba76803bf"
format_stamp: "Formatted at 2026-08-07 11:28:17 on dist-test-slave-2w3w"
server_key: "050974194f600ece444a2d786061f2bc"
server_key_iv: "5ad8532b8db10f9029a4b424bb462292"
server_key_version: "encryptionkey@0"
I20260807 11:28:17.790120 5278 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 11:28:17.793771 5293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:17.795238 5278 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:28:17.795418 5278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "2e060065878846fb8937520ba76803bf"
format_stamp: "Formatted at 2026-08-07 11:28:17 on dist-test-slave-2w3w"
server_key: "050974194f600ece444a2d786061f2bc"
server_key_iv: "5ad8532b8db10f9029a4b424bb462292"
server_key_version: "encryptionkey@0"
I20260807 11:28:17.795631 5278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:17.817094 5278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:17.821594 5278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:17.821895 5278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:17.823007 5278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:28:17.824398 5278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:28:17.824477 5278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:17.824643 5278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:28:17.824704 5278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:17.856688 5278 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:40443
I20260807 11:28:17.856776 5407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:40443 every 8 connection(s)
I20260807 11:28:17.858032 5278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:28:17.865425 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 5278
I20260807 11:28:17.865636 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance
I20260807 11:28:17.866159 17172 external_mini_cluster.cc:1428] Setting key 2f235e33654a24e46e6007524a4bd896
Aug 07 11:28:17 dist-test-slave-2w3w krb5kdc[5169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102097, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.1@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:28:17.880255 5411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:17.860603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:43431 (local address 127.16.197.62:41791)
0807 11:28:17.861062 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:17.861067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:17.861979 (+ 912us) server_negotiation.cc:407] Connection header received
0807 11:28:17.863900 (+ 1921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:17.863905 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:17.864002 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:17.864124 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:17.865658 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:17.866568 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:17.868955 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:17.869427 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:17.869568 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:17.873822 (+ 4254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:17.873827 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:17.873831 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:17.873872 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:17.876574 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:17.877582 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:17.877585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:17.877588 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:17.877710 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:17.878215 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:17.878218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:17.878220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:17.878482 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:17.878664 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:17.879461 (+ 797us) server_negotiation.cc:299] Negotiation successful
0807 11:28:17.879746 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":226,"threads_started":1}
I20260807 11:28:17.880837 5412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:17.861119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:41791 (local address 127.16.197.1:43431)
0807 11:28:17.861812 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:17.861864 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:17.863626 (+ 1762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:17.864364 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:17.864372 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:17.864875 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:28:17.865438 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:17.865455 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:17.866728 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:17.866732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:17.868806 (+ 2074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:17.868819 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:17.869659 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:17.869668 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:17.869954 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:17.870716 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:28:17.870740 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:28:17.873588 (+ 2848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:28:17.876724 (+ 3136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:28:17.876737 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:28:17.876761 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:28:17.877447 (+ 686us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:28:17.877816 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:28:17.877823 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:28:17.877829 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:28:17.878102 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:28:17.878644 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:28:17.878651 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:28:17.879058 (+ 407us) client_negotiation.cc:769] Sending connection context
0807 11:28:17.879295 (+ 237us) client_negotiation.cc:240] Negotiation successful
0807 11:28:17.879707 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":265,"threads_started":1}
I20260807 11:28:17.882436 5408 heartbeater.cc:347] Connected to a master server at 127.16.197.62:41791
I20260807 11:28:17.882776 5408 heartbeater.cc:464] Registering TS with master...
I20260807 11:28:17.883728 5408 heartbeater.cc:511] Master 127.16.197.62:41791 requested a full tablet report, sending...
I20260807 11:28:17.886020 5214 ts_manager.cc:194] Registered new tserver with Master: 2e060065878846fb8937520ba76803bf (127.16.197.1:40443)
I20260807 11:28:17.887566 5214 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.1@KRBTEST.COM'} at 127.16.197.1:43431
WARNING: no policy specified for kudu/127.16.197.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
Entry for principal kudu/127.16.197.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab.
I20260807 11:28:17.933902 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:0
--local_ip_for_outbound_sockets=127.16.197.2
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:41791
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:40925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:18.288805 5417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:18.289155 5417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:18.289335 5417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:18.296846 5417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:18.297009 5417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:18.297204 5417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:28:18.307451 5417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:40925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.2.keytab
--principal=kudu/127.16.197.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:41791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:18.308933 5417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:18.310309 5417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:18.323606 5423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:18.323632 5422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:18.323889 5425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:18.324842 5417 server_base.cc:1034] running on GCE node
I20260807 11:28:18.325646 5417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:18.326601 5417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:18.327847 5417 hybrid_clock.cc:648] HybridClock initialized: now 1786102098327804 us; error 54 us; skew 500 ppm
I20260807 11:28:18.328173 5417 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:18 dist-test-slave-2w3w krb5kdc[5169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102098, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:18.331416 5417 init.cc:377] Logged in from keytab as kudu/127.16.197.2@KRBTEST.COM (short username kudu)
I20260807 11:28:18.332840 5417 webserver.cc:460] Webserver started at http://127.16.197.2:35319/ using document root <none> and password file <none>
I20260807 11:28:18.333467 5417 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:18.333566 5417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:18.333825 5417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:18.336735 5417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-1/data/instance:
uuid: "e8a092ce0de1441c8d973f50fc0d95d3"
format_stamp: "Formatted at 2026-08-07 11:28:18 on dist-test-slave-2w3w"
server_key: "6c6c736843bdbb609f6367732c722c26"
server_key_iv: "e2527d837e8e4e85de1da1bf293d8732"
server_key_version: "encryptionkey@0"
I20260807 11:28:18.337438 5417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance:
uuid: "e8a092ce0de1441c8d973f50fc0d95d3"
format_stamp: "Formatted at 2026-08-07 11:28:18 on dist-test-slave-2w3w"
server_key: "6c6c736843bdbb609f6367732c722c26"
server_key_iv: "e2527d837e8e4e85de1da1bf293d8732"
server_key_version: "encryptionkey@0"
I20260807 11:28:18.342419 5417 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 11:28:18.345876 5433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:18.347263 5417 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:28:18.347421 5417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "e8a092ce0de1441c8d973f50fc0d95d3"
format_stamp: "Formatted at 2026-08-07 11:28:18 on dist-test-slave-2w3w"
server_key: "6c6c736843bdbb609f6367732c722c26"
server_key_iv: "e2527d837e8e4e85de1da1bf293d8732"
server_key_version: "encryptionkey@0"
I20260807 11:28:18.347625 5417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:18.370435 5417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:18.374671 5417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:18.375056 5417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:18.376108 5417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:28:18.377408 5417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:28:18.377488 5417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:18.377619 5417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:28:18.377673 5417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:18.411983 5417 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:43929
I20260807 11:28:18.412055 5551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:43929 every 8 connection(s)
I20260807 11:28:18.413293 5417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:28:18.423496 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 5417
I20260807 11:28:18.423717 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance
I20260807 11:28:18.424172 17172 external_mini_cluster.cc:1428] Setting key 464659426997914ab5494d590658060c
Aug 07 11:28:18 dist-test-slave-2w3w krb5kdc[5169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102098, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.2@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:28:18.434001 5554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:18.415915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:55833 (local address 127.16.197.62:41791)
0807 11:28:18.416365 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:18.416370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:18.417329 (+ 959us) server_negotiation.cc:407] Connection header received
0807 11:28:18.419285 (+ 1956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:18.419293 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:18.419401 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:18.419538 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:18.421025 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:18.421765 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:18.423595 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:18.424164 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:18.424318 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:18.428158 (+ 3840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:18.428171 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:18.428174 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:18.428205 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:18.430739 (+ 2534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:18.431545 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:18.431551 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:18.431554 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:18.431736 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:18.432182 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:18.432186 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:18.432190 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:18.432463 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:18.432619 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:18.433345 (+ 726us) server_negotiation.cc:299] Negotiation successful
0807 11:28:18.433595 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":213,"threads_started":1}
I20260807 11:28:18.435087 5555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:18.416426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:41791 (local address 127.16.197.2:55833)
0807 11:28:18.417175 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:18.417216 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:18.419008 (+ 1792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:18.419772 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:18.419779 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:18.420277 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:28:18.420818 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:18.420839 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:18.421934 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:18.421938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:18.423416 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:18.423431 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:18.424423 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:18.424431 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:18.424670 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:18.425373 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:28:18.425396 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:28:18.427897 (+ 2501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:28:18.430882 (+ 2985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:28:18.430891 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:28:18.430908 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:28:18.431411 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:28:18.431853 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:28:18.431858 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:28:18.431862 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:28:18.432062 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:28:18.432602 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:28:18.432611 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:28:18.432963 (+ 352us) client_negotiation.cc:769] Sending connection context
0807 11:28:18.433236 (+ 273us) client_negotiation.cc:240] Negotiation successful
0807 11:28:18.433825 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":280,"threads_started":1}
I20260807 11:28:18.436647 5552 heartbeater.cc:347] Connected to a master server at 127.16.197.62:41791
I20260807 11:28:18.436964 5552 heartbeater.cc:464] Registering TS with master...
I20260807 11:28:18.437681 5552 heartbeater.cc:511] Master 127.16.197.62:41791 requested a full tablet report, sending...
I20260807 11:28:18.439226 5214 ts_manager.cc:194] Registered new tserver with Master: e8a092ce0de1441c8d973f50fc0d95d3 (127.16.197.2:43929)
I20260807 11:28:18.440521 5214 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.2@KRBTEST.COM'} at 127.16.197.2:55833
WARNING: no policy specified for kudu/127.16.197.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
Entry for principal kudu/127.16.197.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab.
I20260807 11:28:18.490302 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:0
--local_ip_for_outbound_sockets=127.16.197.3
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:41791
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:40925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:18.825508 5559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:18.825811 5559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:18.825922 5559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:18.833191 5559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:18.833303 5559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:18.833415 5559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:28:18.843267 5559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:40925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.3.keytab
--principal=kudu/127.16.197.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:41791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:18.844612 5559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:18.845948 5559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:18.860072 5564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:18.860198 5567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:18.859966 5565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:18.862010 5559 server_base.cc:1034] running on GCE node
I20260807 11:28:18.862706 5559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:18.863762 5559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:18.864989 5559 hybrid_clock.cc:648] HybridClock initialized: now 1786102098864933 us; error 52 us; skew 500 ppm
I20260807 11:28:18.865309 5559 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:18 dist-test-slave-2w3w krb5kdc[5169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102098, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:18.868659 5559 init.cc:377] Logged in from keytab as kudu/127.16.197.3@KRBTEST.COM (short username kudu)
I20260807 11:28:18.870316 5559 webserver.cc:460] Webserver started at http://127.16.197.3:40835/ using document root <none> and password file <none>
I20260807 11:28:18.871097 5559 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:18.871251 5559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:18.871595 5559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:18.874467 5559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-2/data/instance:
uuid: "a176b788004b4a37b2353cf7f4327687"
format_stamp: "Formatted at 2026-08-07 11:28:18 on dist-test-slave-2w3w"
server_key: "eb712848d9b2c98094cdf2f8e909a592"
server_key_iv: "7ac208684738222a6bf777f81e0ce6c9"
server_key_version: "encryptionkey@0"
I20260807 11:28:18.875344 5559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance:
uuid: "a176b788004b4a37b2353cf7f4327687"
format_stamp: "Formatted at 2026-08-07 11:28:18 on dist-test-slave-2w3w"
server_key: "eb712848d9b2c98094cdf2f8e909a592"
server_key_iv: "7ac208684738222a6bf777f81e0ce6c9"
server_key_version: "encryptionkey@0"
I20260807 11:28:18.880481 5559 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 11:28:18.883852 5574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:18.885149 5559 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 11:28:18.885331 5559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "a176b788004b4a37b2353cf7f4327687"
format_stamp: "Formatted at 2026-08-07 11:28:18 on dist-test-slave-2w3w"
server_key: "eb712848d9b2c98094cdf2f8e909a592"
server_key_iv: "7ac208684738222a6bf777f81e0ce6c9"
server_key_version: "encryptionkey@0"
I20260807 11:28:18.885541 5559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:18.891439 5408 heartbeater.cc:503] Master 127.16.197.62:41791 was elected leader, sending a full tablet report...
I20260807 11:28:18.903141 5559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:18.907267 5559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:18.907577 5559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:18.908547 5559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:28:18.909898 5559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:28:18.910193 5559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:18.910389 5559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:28:18.910471 5559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:18.945351 5559 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:38745
I20260807 11:28:18.945406 5692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:38745 every 8 connection(s)
I20260807 11:28:18.946669 5559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:28:18.948614 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 5559
I20260807 11:28:18.948843 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance
I20260807 11:28:18.949437 17172 external_mini_cluster.cc:1428] Setting key c15b0262f398e3aabee7d8d2c3238fb8
Aug 07 11:28:18 dist-test-slave-2w3w krb5kdc[5169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102098, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.3@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:28:18.967890 5695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:18.949525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:59759 (local address 127.16.197.62:41791)
0807 11:28:18.950025 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:18.950030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:18.951304 (+ 1274us) server_negotiation.cc:407] Connection header received
0807 11:28:18.953429 (+ 2125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:18.953433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:18.953509 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:18.953599 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:18.955095 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:18.956008 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:18.957798 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:18.958182 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:18.958275 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:18.961792 (+ 3517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:18.961797 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:18.961800 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:18.961830 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:18.964803 (+ 2973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:18.965700 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:18.965704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:18.965706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:18.965831 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:18.966219 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:18.966222 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:18.966224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:18.966529 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:18.966717 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:18.967444 (+ 727us) server_negotiation.cc:299] Negotiation successful
0807 11:28:18.967615 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":242,"threads_started":1}
I20260807 11:28:18.968901 5696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:18.950184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:41791 (local address 127.16.197.3:59759)
0807 11:28:18.951125 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:18.951173 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:18.953186 (+ 2013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:18.953807 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:18.953815 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:18.954315 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:28:18.954874 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:18.954892 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:18.956310 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:18.956314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:18.957657 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:18.957670 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:18.958386 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:18.958391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:18.958602 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:18.959399 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:28:18.959426 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:28:18.961595 (+ 2169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:28:18.964968 (+ 3373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:28:18.964977 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:28:18.964995 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:28:18.965585 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:28:18.965926 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:28:18.965930 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:28:18.965932 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:28:18.966121 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:28:18.966695 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:28:18.966703 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:28:18.967109 (+ 406us) client_negotiation.cc:769] Sending connection context
0807 11:28:18.967343 (+ 234us) client_negotiation.cc:240] Negotiation successful
0807 11:28:18.967747 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":322,"threads_started":1}
I20260807 11:28:18.970443 5693 heartbeater.cc:347] Connected to a master server at 127.16.197.62:41791
I20260807 11:28:18.970759 5693 heartbeater.cc:464] Registering TS with master...
I20260807 11:28:18.971575 5693 heartbeater.cc:511] Master 127.16.197.62:41791 requested a full tablet report, sending...
I20260807 11:28:18.973034 5214 ts_manager.cc:194] Registered new tserver with Master: a176b788004b4a37b2353cf7f4327687 (127.16.197.3:38745)
I20260807 11:28:18.973809 5214 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.197.3@KRBTEST.COM'} at 127.16.197.3:59759
I20260807 11:28:18.977494 17172 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 11:28:18.994354 5695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:18.981488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36442 (local address 127.16.197.62:41791)
0807 11:28:18.981668 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:18.981676 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:18.982108 (+ 432us) server_negotiation.cc:407] Connection header received
0807 11:28:18.982310 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:18.982315 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:18.982378 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:18.982484 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:18.984013 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:18.984746 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:18.986035 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:18.986458 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:18.986593 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:18.989093 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:18.989098 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:18.989101 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:18.989134 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:18.991692 (+ 2558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:18.992314 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:18.992322 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:18.992325 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:18.992455 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:18.992836 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:18.992840 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:18.992845 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:18.993170 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:18.993424 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:18.993793 (+ 369us) server_negotiation.cc:299] Negotiation successful
0807 11:28:18.994040 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 11:28:18.998109 5214 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36442:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 11:28:19.005617 5214 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:28:19.029260 5712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:19.018260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:38745 (local address 127.0.0.1:41220)
0807 11:28:19.020979 (+ 2719us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:19.020997 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:19.021144 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:19.021790 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:19.021797 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:19.021839 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:19.022402 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:19.022415 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:19.024470 (+ 2055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:19.024475 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:19.026172 (+ 1697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:19.026185 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:19.028129 (+ 1944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:19.028137 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:19.028407 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:19.028487 (+ 80us) client_negotiation.cc:769] Sending connection context
0807 11:28:19.028607 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 11:28:19.028790 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2565,"mutex_wait_us":50,"spinlock_wait_cycles":896,"thread_start_us":2542,"threads_started":1}
I20260807 11:28:19.029520 5710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:19.018746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41220 (local address 127.16.197.3:38745)
0807 11:28:19.019734 (+ 988us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:19.019743 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:19.021101 (+ 1358us) server_negotiation.cc:407] Connection header received
0807 11:28:19.021362 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:19.021368 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:19.021570 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:19.021770 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:19.022634 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:19.024332 (+ 1698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:19.026391 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:19.027901 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:19.028191 (+ 290us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:19.028628 (+ 437us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:19.028739 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 11:28:19.028992 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":804,"thread_start_us":801,"threads_started":1}
I20260807 11:28:19.031548 5708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:19.017942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45124 (local address 127.16.197.1:40443)
0807 11:28:19.019496 (+ 1554us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:19.019509 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:19.020903 (+ 1394us) server_negotiation.cc:407] Connection header received
0807 11:28:19.021311 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:19.021320 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:19.021572 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:19.021770 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:19.022628 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:19.024321 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:19.028416 (+ 4095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:19.030077 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:19.030218 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:19.030452 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:19.031018 (+ 566us) server_negotiation.cc:299] Negotiation successful
0807 11:28:19.031201 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1306,"thread_start_us":1123,"threads_started":1}
I20260807 11:28:19.032197 5709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:19.019032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:43929 (local address 127.0.0.1:58984)
0807 11:28:19.021113 (+ 2081us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:19.021128 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:19.021242 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:19.022477 (+ 1235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:19.022482 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:19.022505 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:19.023218 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:19.023229 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:19.024831 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:19.024834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:19.026371 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:19.026385 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:19.030216 (+ 3831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:19.030223 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:19.030387 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:19.030442 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 11:28:19.031668 (+ 1226us) client_negotiation.cc:240] Negotiation successful
0807 11:28:19.031809 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1964,"mutex_wait_us":2,"thread_start_us":1586,"threads_started":1}
I20260807 11:28:19.032634 5707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:19.017755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:40443 (local address 127.0.0.1:45124)
0807 11:28:19.020703 (+ 2948us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:19.020756 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:19.021066 (+ 310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:19.021790 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:19.021797 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:19.021846 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:19.022402 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:19.022415 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:19.026395 (+ 3980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:19.026401 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:19.028223 (+ 1822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:19.028235 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:19.030583 (+ 2348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:19.030588 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:19.030759 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:19.030796 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 11:28:19.031953 (+ 1157us) client_negotiation.cc:240] Negotiation successful
0807 11:28:19.032080 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2340,"mutex_wait_us":36,"spinlock_wait_cycles":5632,"thread_start_us":366,"threads_started":1}
I20260807 11:28:19.033259 5711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:19.019315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58984 (local address 127.16.197.2:43929)
0807 11:28:19.020012 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:19.020022 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:19.022051 (+ 2029us) server_negotiation.cc:407] Connection header received
0807 11:28:19.022132 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:19.022137 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:19.022288 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:19.022474 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:19.023402 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:19.024686 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:19.026571 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:19.027983 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:19.032587 (+ 4604us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:19.032810 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:19.032877 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 11:28:19.033022 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":351,"threads_started":1}
I20260807 11:28:19.034512 5624 tablet_service.cc:1467] Processing CreateTablet for tablet 2dc0dfe3f1824f768cba4b41b0019276 (DEFAULT_TABLE table=test-table [id=95356873acf94888b18dd512fb411282]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:28:19.036202 5624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dc0dfe3f1824f768cba4b41b0019276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:19.036283 5342 tablet_service.cc:1467] Processing CreateTablet for tablet 2dc0dfe3f1824f768cba4b41b0019276 (DEFAULT_TABLE table=test-table [id=95356873acf94888b18dd512fb411282]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:28:19.036983 5483 tablet_service.cc:1467] Processing CreateTablet for tablet 2dc0dfe3f1824f768cba4b41b0019276 (DEFAULT_TABLE table=test-table [id=95356873acf94888b18dd512fb411282]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:28:19.037755 5342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dc0dfe3f1824f768cba4b41b0019276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:19.038298 5483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dc0dfe3f1824f768cba4b41b0019276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:19.051949 5714 tablet_bootstrap.cc:492] T 2dc0dfe3f1824f768cba4b41b0019276 P a176b788004b4a37b2353cf7f4327687: Bootstrap starting.
I20260807 11:28:19.055891 5713 tablet_bootstrap.cc:492] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf: Bootstrap starting.
I20260807 11:28:19.056577 5714 tablet_bootstrap.cc:654] T 2dc0dfe3f1824f768cba4b41b0019276 P a176b788004b4a37b2353cf7f4327687: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:19.057678 5714 log.cc:824] T 2dc0dfe3f1824f768cba4b41b0019276 P a176b788004b4a37b2353cf7f4327687: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:19.060639 5713 tablet_bootstrap.cc:654] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:19.062155 5713 log.cc:824] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:19.063480 5715 tablet_bootstrap.cc:492] T 2dc0dfe3f1824f768cba4b41b0019276 P e8a092ce0de1441c8d973f50fc0d95d3: Bootstrap starting.
I20260807 11:28:19.066107 5714 tablet_bootstrap.cc:492] T 2dc0dfe3f1824f768cba4b41b0019276 P a176b788004b4a37b2353cf7f4327687: No bootstrap required, opened a new log
I20260807 11:28:19.066367 5714 ts_tablet_manager.cc:1397] T 2dc0dfe3f1824f768cba4b41b0019276 P a176b788004b4a37b2353cf7f4327687: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.008s
I20260807 11:28:19.066475 5715 tablet_bootstrap.cc:654] T 2dc0dfe3f1824f768cba4b41b0019276 P e8a092ce0de1441c8d973f50fc0d95d3: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:19.067636 5715 log.cc:824] T 2dc0dfe3f1824f768cba4b41b0019276 P e8a092ce0de1441c8d973f50fc0d95d3: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:19.069334 5713 tablet_bootstrap.cc:492] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf: No bootstrap required, opened a new log
I20260807 11:28:19.069681 5713 ts_tablet_manager.cc:1397] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.008s
I20260807 11:28:19.071069 5715 tablet_bootstrap.cc:492] T 2dc0dfe3f1824f768cba4b41b0019276 P e8a092ce0de1441c8d973f50fc0d95d3: No bootstrap required, opened a new log
I20260807 11:28:19.071329 5715 ts_tablet_manager.cc:1397] T 2dc0dfe3f1824f768cba4b41b0019276 P e8a092ce0de1441c8d973f50fc0d95d3: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260807 11:28:19.075181 5714 raft_consensus.cc:348] T 2dc0dfe3f1824f768cba4b41b0019276 P a176b788004b4a37b2353cf7f4327687 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e060065878846fb8937520ba76803bf" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 40443 } } peers { permanent_uuid: "e8a092ce0de1441c8d973f50fc0d95d3" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 43929 } } peers { permanent_uuid: "a176b788004b4a37b2353cf7f4327687" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 38745 } }
I20260807 11:28:19.075548 5714 raft_consensus.cc:374] T 2dc0dfe3f1824f768cba4b41b0019276 P a176b788004b4a37b2353cf7f4327687 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:19.075659 5714 raft_consensus.cc:729] T 2dc0dfe3f1824f768cba4b41b0019276 P a176b788004b4a37b2353cf7f4327687 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a176b788004b4a37b2353cf7f4327687, State: Initialized, Role: FOLLOWER
I20260807 11:28:19.076269 5714 consensus_queue.cc:260] T 2dc0dfe3f1824f768cba4b41b0019276 P a176b788004b4a37b2353cf7f4327687 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e060065878846fb8937520ba76803bf" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 40443 } } peers { permanent_uuid: "e8a092ce0de1441c8d973f50fc0d95d3" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 43929 } } peers { permanent_uuid: "a176b788004b4a37b2353cf7f4327687" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 38745 } }
I20260807 11:28:19.077363 5693 heartbeater.cc:503] Master 127.16.197.62:41791 was elected leader, sending a full tablet report...
I20260807 11:28:19.077870 5714 ts_tablet_manager.cc:1428] T 2dc0dfe3f1824f768cba4b41b0019276 P a176b788004b4a37b2353cf7f4327687: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260807 11:28:19.080397 5713 raft_consensus.cc:348] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e060065878846fb8937520ba76803bf" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 40443 } } peers { permanent_uuid: "e8a092ce0de1441c8d973f50fc0d95d3" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 43929 } } peers { permanent_uuid: "a176b788004b4a37b2353cf7f4327687" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 38745 } }
I20260807 11:28:19.080858 5713 raft_consensus.cc:374] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:19.080969 5713 raft_consensus.cc:729] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e060065878846fb8937520ba76803bf, State: Initialized, Role: FOLLOWER
I20260807 11:28:19.081583 5713 consensus_queue.cc:260] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e060065878846fb8937520ba76803bf" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 40443 } } peers { permanent_uuid: "e8a092ce0de1441c8d973f50fc0d95d3" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 43929 } } peers { permanent_uuid: "a176b788004b4a37b2353cf7f4327687" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 38745 } }
I20260807 11:28:19.081954 5715 raft_consensus.cc:348] T 2dc0dfe3f1824f768cba4b41b0019276 P e8a092ce0de1441c8d973f50fc0d95d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e060065878846fb8937520ba76803bf" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 40443 } } peers { permanent_uuid: "e8a092ce0de1441c8d973f50fc0d95d3" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 43929 } } peers { permanent_uuid: "a176b788004b4a37b2353cf7f4327687" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 38745 } }
I20260807 11:28:19.082365 5715 raft_consensus.cc:374] T 2dc0dfe3f1824f768cba4b41b0019276 P e8a092ce0de1441c8d973f50fc0d95d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:19.082463 5715 raft_consensus.cc:729] T 2dc0dfe3f1824f768cba4b41b0019276 P e8a092ce0de1441c8d973f50fc0d95d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8a092ce0de1441c8d973f50fc0d95d3, State: Initialized, Role: FOLLOWER
I20260807 11:28:19.082989 5715 consensus_queue.cc:260] T 2dc0dfe3f1824f768cba4b41b0019276 P e8a092ce0de1441c8d973f50fc0d95d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e060065878846fb8937520ba76803bf" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 40443 } } peers { permanent_uuid: "e8a092ce0de1441c8d973f50fc0d95d3" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 43929 } } peers { permanent_uuid: "a176b788004b4a37b2353cf7f4327687" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 38745 } }
I20260807 11:28:19.084578 5552 heartbeater.cc:503] Master 127.16.197.62:41791 was elected leader, sending a full tablet report...
I20260807 11:28:19.085326 5715 ts_tablet_manager.cc:1428] T 2dc0dfe3f1824f768cba4b41b0019276 P e8a092ce0de1441c8d973f50fc0d95d3: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260807 11:28:19.086777 5713 ts_tablet_manager.cc:1428] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf: Time spent starting tablet: real 0.017s user 0.013s sys 0.002s
I20260807 11:28:19.099948 5721 raft_consensus.cc:482] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:28:19.100248 5721 raft_consensus.cc:504] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e060065878846fb8937520ba76803bf" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 40443 } } peers { permanent_uuid: "e8a092ce0de1441c8d973f50fc0d95d3" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 43929 } } peers { permanent_uuid: "a176b788004b4a37b2353cf7f4327687" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 38745 } }
I20260807 11:28:19.101969 5721 leader_election.cc:288] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8a092ce0de1441c8d973f50fc0d95d3 (127.16.197.2:43929), a176b788004b4a37b2353cf7f4327687 (127.16.197.3:38745)
I20260807 11:28:19.108759 5710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:19.102536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:43109 (local address 127.16.197.3:38745)
0807 11:28:19.102748 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:19.102755 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:19.103421 (+ 666us) server_negotiation.cc:407] Connection header received
0807 11:28:19.103637 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:19.103645 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:19.103711 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:19.103811 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:19.104426 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:19.105188 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:19.107039 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:19.107799 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:19.107990 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:19.108071 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:19.108375 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 11:28:19.108498 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 11:28:19.108785 5711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:19.102424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:34259 (local address 127.16.197.2:43929)
0807 11:28:19.102649 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:19.102656 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:19.103556 (+ 900us) server_negotiation.cc:407] Connection header received
0807 11:28:19.103755 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:19.103759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:19.103816 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:19.103910 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:19.104559 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:19.105252 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:19.107094 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:19.107856 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:19.108042 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:19.108123 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:19.108397 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 11:28:19.108529 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 11:28:19.108928 5723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:19.102414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:38745 (local address 127.16.197.1:43109)
0807 11:28:19.103311 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:19.103331 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:19.103500 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:19.103827 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:19.103832 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:19.103874 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:19.104270 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:19.104283 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:19.105318 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:19.105326 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:19.106868 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:19.106880 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:19.108045 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:19.108053 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:19.108205 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:19.108229 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 11:28:19.108333 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 11:28:19.108460 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"mutex_wait_us":8,"thread_start_us":526,"threads_started":1}
I20260807 11:28:19.108951 5724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:19.102409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:43929 (local address 127.16.197.1:34259)
0807 11:28:19.103444 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:19.103459 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:19.103615 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:19.103930 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:19.103935 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:19.103976 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:19.104400 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:19.104409 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:19.105391 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:19.105396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:19.106893 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:19.106905 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:19.108093 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:19.108099 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:19.108244 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:19.108267 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 11:28:19.108371 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 11:28:19.108502 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":914,"mutex_wait_us":45,"spinlock_wait_cycles":20352,"thread_start_us":764,"threads_started":1}
I20260807 11:28:19.110055 5504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dc0dfe3f1824f768cba4b41b0019276" candidate_uuid: "2e060065878846fb8937520ba76803bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8a092ce0de1441c8d973f50fc0d95d3" is_pre_election: true
I20260807 11:28:19.110055 5645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dc0dfe3f1824f768cba4b41b0019276" candidate_uuid: "2e060065878846fb8937520ba76803bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a176b788004b4a37b2353cf7f4327687" is_pre_election: true
W20260807 11:28:19.110615 5410 tablet.cc:2369] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:28:19.110644 5504 raft_consensus.cc:2454] T 2dc0dfe3f1824f768cba4b41b0019276 P e8a092ce0de1441c8d973f50fc0d95d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e060065878846fb8937520ba76803bf in term 0.
I20260807 11:28:19.110644 5645 raft_consensus.cc:2454] T 2dc0dfe3f1824f768cba4b41b0019276 P a176b788004b4a37b2353cf7f4327687 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e060065878846fb8937520ba76803bf in term 0.
I20260807 11:28:19.111454 5296 leader_election.cc:302] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e060065878846fb8937520ba76803bf, a176b788004b4a37b2353cf7f4327687; no voters:
I20260807 11:28:19.111889 5721 raft_consensus.cc:2781] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:28:19.112043 5721 raft_consensus.cc:482] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:28:19.112193 5721 raft_consensus.cc:3037] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:19.114365 5721 raft_consensus.cc:504] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e060065878846fb8937520ba76803bf" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 40443 } } peers { permanent_uuid: "e8a092ce0de1441c8d973f50fc0d95d3" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 43929 } } peers { permanent_uuid: "a176b788004b4a37b2353cf7f4327687" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 38745 } }
I20260807 11:28:19.115067 5721 leader_election.cc:288] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf [CANDIDATE]: Term 1 election: Requested vote from peers e8a092ce0de1441c8d973f50fc0d95d3 (127.16.197.2:43929), a176b788004b4a37b2353cf7f4327687 (127.16.197.3:38745)
I20260807 11:28:19.115460 5504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dc0dfe3f1824f768cba4b41b0019276" candidate_uuid: "2e060065878846fb8937520ba76803bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8a092ce0de1441c8d973f50fc0d95d3"
I20260807 11:28:19.115553 5645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dc0dfe3f1824f768cba4b41b0019276" candidate_uuid: "2e060065878846fb8937520ba76803bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a176b788004b4a37b2353cf7f4327687"
I20260807 11:28:19.115682 5504 raft_consensus.cc:3037] T 2dc0dfe3f1824f768cba4b41b0019276 P e8a092ce0de1441c8d973f50fc0d95d3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:19.115814 5645 raft_consensus.cc:3037] T 2dc0dfe3f1824f768cba4b41b0019276 P a176b788004b4a37b2353cf7f4327687 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:19.117969 5504 raft_consensus.cc:2454] T 2dc0dfe3f1824f768cba4b41b0019276 P e8a092ce0de1441c8d973f50fc0d95d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e060065878846fb8937520ba76803bf in term 1.
I20260807 11:28:19.118078 5645 raft_consensus.cc:2454] T 2dc0dfe3f1824f768cba4b41b0019276 P a176b788004b4a37b2353cf7f4327687 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e060065878846fb8937520ba76803bf in term 1.
I20260807 11:28:19.118363 5297 leader_election.cc:302] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e060065878846fb8937520ba76803bf, e8a092ce0de1441c8d973f50fc0d95d3; no voters:
I20260807 11:28:19.118683 5721 raft_consensus.cc:2781] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:19.119242 5721 raft_consensus.cc:686] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf [term 1 LEADER]: Becoming Leader. State: Replica: 2e060065878846fb8937520ba76803bf, State: Running, Role: LEADER
I20260807 11:28:19.119766 5721 consensus_queue.cc:237] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e060065878846fb8937520ba76803bf" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 40443 } } peers { permanent_uuid: "e8a092ce0de1441c8d973f50fc0d95d3" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 43929 } } peers { permanent_uuid: "a176b788004b4a37b2353cf7f4327687" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 38745 } }
I20260807 11:28:19.124405 5213 catalog_manager.cc:5495] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e060065878846fb8937520ba76803bf (127.16.197.1). New cstate: current_term: 1 leader_uuid: "2e060065878846fb8937520ba76803bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e060065878846fb8937520ba76803bf" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 40443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8a092ce0de1441c8d973f50fc0d95d3" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 43929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a176b788004b4a37b2353cf7f4327687" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 38745 } health_report { overall_health: UNKNOWN } } }
I20260807 11:28:19.155429 5708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:19.150166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45138 (local address 127.16.197.1:40443)
0807 11:28:19.150415 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:19.150420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:19.150441 (+ 21us) server_negotiation.cc:407] Connection header received
0807 11:28:19.150639 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:19.150643 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:19.150723 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:19.150900 (+ 177us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:28:19.151306 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:19.152085 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:19.153406 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:19.153805 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:19.153895 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:19.154324 (+ 429us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:28:19.154841 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:28:19.155020 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:19.155084 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 11:28:19.155150 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 11:28:19.164014 5504 raft_consensus.cc:1261] T 2dc0dfe3f1824f768cba4b41b0019276 P e8a092ce0de1441c8d973f50fc0d95d3 [term 1 FOLLOWER]: Refusing update from remote peer 2e060065878846fb8937520ba76803bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:28:19.164014 5645 raft_consensus.cc:1261] T 2dc0dfe3f1824f768cba4b41b0019276 P a176b788004b4a37b2353cf7f4327687 [term 1 FOLLOWER]: Refusing update from remote peer 2e060065878846fb8937520ba76803bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:28:19.165019 5725 consensus_queue.cc:1035] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a176b788004b4a37b2353cf7f4327687" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 38745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 11:28:19.165373 5553 tablet.cc:2369] T 2dc0dfe3f1824f768cba4b41b0019276 P e8a092ce0de1441c8d973f50fc0d95d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:28:19.165781 5721 consensus_queue.cc:1035] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8a092ce0de1441c8d973f50fc0d95d3" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 43929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:28:19.198446 5213 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:28:19.198760 5213 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:28:19.202674 5213 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c8df210e8594ddca88be898b00eb18a: Sending DeleteTablet for 3 replicas of tablet 2dc0dfe3f1824f768cba4b41b0019276
I20260807 11:28:19.203913 5342 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dc0dfe3f1824f768cba4b41b0019276 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:28:19 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:45124
I20260807 11:28:19.204056 5624 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dc0dfe3f1824f768cba4b41b0019276 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:28:19 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:41220
I20260807 11:28:19.204056 5483 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dc0dfe3f1824f768cba4b41b0019276 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:28:19 UTC) from {username='kudu', principal='kudu/127.16.197.62@KRBTEST.COM'} at 127.0.0.1:58984
I20260807 11:28:19.209115 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 5278
I20260807 11:28:19.216928 5739 tablet_replica.cc:330] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf: stopping tablet replica
I20260807 11:28:19.217504 5739 raft_consensus.cc:2229] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf [term 1 LEADER]: Raft consensus shutting down.
I20260807 11:28:19.218262 5739 raft_consensus.cc:2258] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:28:19.222834 5739 ts_tablet_manager.cc:1905] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:28:19.232028 5740 tablet_replica.cc:330] T 2dc0dfe3f1824f768cba4b41b0019276 P e8a092ce0de1441c8d973f50fc0d95d3: stopping tablet replica
I20260807 11:28:19.232544 5740 raft_consensus.cc:2229] T 2dc0dfe3f1824f768cba4b41b0019276 P e8a092ce0de1441c8d973f50fc0d95d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:28:19.232637 5739 ts_tablet_manager.cc:1918] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:28:19.232824 5739 log.cc:1196] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-0/wal/wals/2dc0dfe3f1824f768cba4b41b0019276
I20260807 11:28:19.233162 5740 raft_consensus.cc:2258] T 2dc0dfe3f1824f768cba4b41b0019276 P e8a092ce0de1441c8d973f50fc0d95d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:28:19.233457 5739 ts_tablet_manager.cc:1939] T 2dc0dfe3f1824f768cba4b41b0019276 P 2e060065878846fb8937520ba76803bf: Deleting consensus metadata
I20260807 11:28:19.235458 5201 catalog_manager.cc:4841] TS 2e060065878846fb8937520ba76803bf (127.16.197.1:40443): tablet 2dc0dfe3f1824f768cba4b41b0019276 (table test-table [id=95356873acf94888b18dd512fb411282]) successfully deleted
I20260807 11:28:19.238095 5741 tablet_replica.cc:330] T 2dc0dfe3f1824f768cba4b41b0019276 P a176b788004b4a37b2353cf7f4327687: stopping tablet replica
I20260807 11:28:19.238586 5741 raft_consensus.cc:2229] T 2dc0dfe3f1824f768cba4b41b0019276 P a176b788004b4a37b2353cf7f4327687 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:28:19.239176 5741 raft_consensus.cc:2258] T 2dc0dfe3f1824f768cba4b41b0019276 P a176b788004b4a37b2353cf7f4327687 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:28:19.241631 5740 ts_tablet_manager.cc:1905] T 2dc0dfe3f1824f768cba4b41b0019276 P e8a092ce0de1441c8d973f50fc0d95d3: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 11:28:19 dist-test-slave-2w3w krb5kdc[5169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.1@KRBTEST.COM
I20260807 11:28:19.244532 5741 ts_tablet_manager.cc:1905] T 2dc0dfe3f1824f768cba4b41b0019276 P a176b788004b4a37b2353cf7f4327687: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:28:19.248461 5740 ts_tablet_manager.cc:1918] T 2dc0dfe3f1824f768cba4b41b0019276 P e8a092ce0de1441c8d973f50fc0d95d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:28:19.248629 5740 log.cc:1196] T 2dc0dfe3f1824f768cba4b41b0019276 P e8a092ce0de1441c8d973f50fc0d95d3: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-1/wal/wals/2dc0dfe3f1824f768cba4b41b0019276
I20260807 11:28:19.249293 5740 ts_tablet_manager.cc:1939] T 2dc0dfe3f1824f768cba4b41b0019276 P e8a092ce0de1441c8d973f50fc0d95d3: Deleting consensus metadata
I20260807 11:28:19.251225 5201 catalog_manager.cc:4841] TS e8a092ce0de1441c8d973f50fc0d95d3 (127.16.197.2:43929): tablet 2dc0dfe3f1824f768cba4b41b0019276 (table test-table [id=95356873acf94888b18dd512fb411282]) successfully deleted
I20260807 11:28:19.252472 5708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:19.210488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45152 (local address 127.16.197.1:40443)
0807 11:28:19.210725 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:19.210733 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:19.228698 (+ 17965us) server_negotiation.cc:407] Connection header received
0807 11:28:19.229257 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:19.229263 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:19.229358 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:19.229456 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:19.233621 (+ 4165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:19.234863 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:19.236847 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:19.237759 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:19.237901 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:19.244053 (+ 6152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:19.244067 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:19.244087 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:19.244139 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:19.248333 (+ 4194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:19.249363 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:19.249374 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:19.249383 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:19.249575 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:19.250076 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:19.250082 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:19.250085 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:19.250566 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:19.250919 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:19.251617 (+ 698us) server_negotiation.cc:299] Negotiation successful
0807 11:28:19.251900 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 11:28:19.252966 5741 ts_tablet_manager.cc:1918] T 2dc0dfe3f1824f768cba4b41b0019276 P a176b788004b4a37b2353cf7f4327687: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:28:19.253134 5741 log.cc:1196] T 2dc0dfe3f1824f768cba4b41b0019276 P a176b788004b4a37b2353cf7f4327687: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786101765888975-17172-0/minicluster-data/ts-2/wal/wals/2dc0dfe3f1824f768cba4b41b0019276
I20260807 11:28:19.253261 5403 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:19.253700 5741 ts_tablet_manager.cc:1939] T 2dc0dfe3f1824f768cba4b41b0019276 P a176b788004b4a37b2353cf7f4327687: Deleting consensus metadata
I20260807 11:28:19.255738 5200 catalog_manager.cc:4841] TS a176b788004b4a37b2353cf7f4327687 (127.16.197.3:38745): tablet 2dc0dfe3f1824f768cba4b41b0019276 (table test-table [id=95356873acf94888b18dd512fb411282]) successfully deleted
I20260807 11:28:19.354765 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 5278
I20260807 11:28:19.375818 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 5417
Aug 07 11:28:19 dist-test-slave-2w3w krb5kdc[5169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.2@KRBTEST.COM
I20260807 11:28:19.391634 5711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:19.376618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58988 (local address 127.16.197.2:43929)
0807 11:28:19.376790 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:19.376798 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:19.376834 (+ 36us) server_negotiation.cc:407] Connection header received
0807 11:28:19.377056 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:19.377060 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:19.377133 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:19.377236 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:19.378534 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:19.379343 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:19.380666 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:19.381073 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:19.381211 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:19.385636 (+ 4425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:19.385650 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:19.385665 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:19.385703 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:19.388714 (+ 3011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:19.389413 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:19.389429 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:19.389432 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:19.389558 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:19.390032 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:19.390039 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:19.390044 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:19.390423 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:19.390729 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:19.391060 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 11:28:19.391286 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 11:28:19.392274 5546 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:19.482276 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 5417
I20260807 11:28:19.501117 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 5559
Aug 07 11:28:19 dist-test-slave-2w3w krb5kdc[5169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.3@KRBTEST.COM
I20260807 11:28:19.516250 5710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:19.501892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41230 (local address 127.16.197.3:38745)
0807 11:28:19.502081 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:19.502086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:19.502104 (+ 18us) server_negotiation.cc:407] Connection header received
0807 11:28:19.502236 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:19.502240 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:19.502305 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:19.502399 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:19.503809 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:19.504577 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:19.505855 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:19.506242 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:19.506335 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:19.510778 (+ 4443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:19.510792 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:19.510808 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:19.510844 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:19.513703 (+ 2859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:19.514305 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:19.514312 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:19.514314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:19.514472 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:19.514815 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:19.514818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:19.514820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:19.515218 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:19.515441 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:19.515709 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 11:28:19.515878 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 11:28:19.516934 5688 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:19.606395 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 5559
I20260807 11:28:19.625298 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 5182
I20260807 11:28:19.626413 5246 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:19.744392 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 5182
2026-08-07T11:28:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4433 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T11:28:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:28:19Z Disabled control of system clock
I20260807 11:28:19.831110 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:40977
--webserver_interface=127.16.197.62
--webserver_port=0
--builtin_ntp_servers=127.16.197.20:33829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:40977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 11:28:20.174804 5750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:20.175124 5750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:20.175299 5750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:20.182557 5750 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:28:20.182675 5750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:20.182755 5750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:20.182848 5750 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:28:20.182945 5750 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:28:20.192256 5750 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:33829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:40977
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:40977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:20.193648 5750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:20.195533 5750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:20.207819 5756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:20.207775 5759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:20.207856 5757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:20.208616 5750 server_base.cc:1034] running on GCE node
I20260807 11:28:20.209396 5750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:20.210764 5750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:20.211999 5750 hybrid_clock.cc:648] HybridClock initialized: now 1786102100211958 us; error 52 us; skew 500 ppm
I20260807 11:28:20.212354 5750 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 11:28:20.214560 5750 webserver.cc:460] Webserver started at http://127.16.197.62:46767/ using document root <none> and password file <none>
I20260807 11:28:20.215315 5750 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:20.215453 5750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:20.215767 5750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:20.218637 5750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "a706aef041b44cfcb97fb4b4a75b7cb6"
format_stamp: "Formatted at 2026-08-07 11:28:20 on dist-test-slave-2w3w"
server_key: "0ffd19cb2e6ee9117a0e8d1dff8c639e"
server_key_iv: "598f7b1c67b4f1e2a0c46e3c48ea0037"
server_key_version: "encryptionkey@0"
I20260807 11:28:20.219496 5750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "a706aef041b44cfcb97fb4b4a75b7cb6"
format_stamp: "Formatted at 2026-08-07 11:28:20 on dist-test-slave-2w3w"
server_key: "0ffd19cb2e6ee9117a0e8d1dff8c639e"
server_key_iv: "598f7b1c67b4f1e2a0c46e3c48ea0037"
server_key_version: "encryptionkey@0"
I20260807 11:28:20.224568 5750 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 11:28:20.228072 5767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:20.229306 5750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 11:28:20.229509 5750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "a706aef041b44cfcb97fb4b4a75b7cb6"
format_stamp: "Formatted at 2026-08-07 11:28:20 on dist-test-slave-2w3w"
server_key: "0ffd19cb2e6ee9117a0e8d1dff8c639e"
server_key_iv: "598f7b1c67b4f1e2a0c46e3c48ea0037"
server_key_version: "encryptionkey@0"
I20260807 11:28:20.229733 5750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:20.243819 5750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:20.244626 5750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:20.244995 5750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:20.263801 5750 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:40977
I20260807 11:28:20.263821 5819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:40977 every 8 connection(s)
I20260807 11:28:20.265194 5750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:28:20.266912 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 5750
I20260807 11:28:20.267144 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:28:20.267731 17172 external_mini_cluster.cc:1428] Setting key 25d733e10444c33b5024a737d5a649b4
I20260807 11:28:20.269793 5820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:20.282693 5823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:20.269669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59982 (local address 127.16.197.62:40977)
0807 11:28:20.270740 (+ 1071us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:20.270754 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:20.270806 (+ 52us) server_negotiation.cc:407] Connection header received
0807 11:28:20.272904 (+ 2098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:20.272930 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:20.273521 (+ 591us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:20.273996 (+ 475us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:20.274532 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:20.276663 (+ 2131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:20.277983 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:20.279046 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:20.279254 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:20.280444 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:20.280451 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:20.280464 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:20.281026 (+ 562us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:20.281106 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:20.281126 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:20.281268 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:20.281498 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 11:28:20.281822 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":384,"threads_started":1}
I20260807 11:28:20.283421 5820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a706aef041b44cfcb97fb4b4a75b7cb6: Bootstrap starting.
I20260807 11:28:20.286515 5820 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a706aef041b44cfcb97fb4b4a75b7cb6: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:20.287582 5820 log.cc:824] T 00000000000000000000000000000000 P a706aef041b44cfcb97fb4b4a75b7cb6: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:20.290310 5820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a706aef041b44cfcb97fb4b4a75b7cb6: No bootstrap required, opened a new log
I20260807 11:28:20.298169 5820 raft_consensus.cc:348] T 00000000000000000000000000000000 P a706aef041b44cfcb97fb4b4a75b7cb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a706aef041b44cfcb97fb4b4a75b7cb6" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 40977 } }
I20260807 11:28:20.298571 5820 raft_consensus.cc:374] T 00000000000000000000000000000000 P a706aef041b44cfcb97fb4b4a75b7cb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:20.298695 5820 raft_consensus.cc:729] T 00000000000000000000000000000000 P a706aef041b44cfcb97fb4b4a75b7cb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a706aef041b44cfcb97fb4b4a75b7cb6, State: Initialized, Role: FOLLOWER
I20260807 11:28:20.299330 5820 consensus_queue.cc:260] T 00000000000000000000000000000000 P a706aef041b44cfcb97fb4b4a75b7cb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a706aef041b44cfcb97fb4b4a75b7cb6" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 40977 } }
I20260807 11:28:20.299584 5820 raft_consensus.cc:388] T 00000000000000000000000000000000 P a706aef041b44cfcb97fb4b4a75b7cb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:28:20.299732 5820 raft_consensus.cc:482] T 00000000000000000000000000000000 P a706aef041b44cfcb97fb4b4a75b7cb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:28:20.299914 5820 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a706aef041b44cfcb97fb4b4a75b7cb6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:20.302073 5820 raft_consensus.cc:504] T 00000000000000000000000000000000 P a706aef041b44cfcb97fb4b4a75b7cb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a706aef041b44cfcb97fb4b4a75b7cb6" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 40977 } }
I20260807 11:28:20.302587 5820 leader_election.cc:302] T 00000000000000000000000000000000 P a706aef041b44cfcb97fb4b4a75b7cb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a706aef041b44cfcb97fb4b4a75b7cb6; no voters:
I20260807 11:28:20.303278 5820 leader_election.cc:288] T 00000000000000000000000000000000 P a706aef041b44cfcb97fb4b4a75b7cb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:28:20.303433 5825 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a706aef041b44cfcb97fb4b4a75b7cb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:20.304816 5825 raft_consensus.cc:686] T 00000000000000000000000000000000 P a706aef041b44cfcb97fb4b4a75b7cb6 [term 1 LEADER]: Becoming Leader. State: Replica: a706aef041b44cfcb97fb4b4a75b7cb6, State: Running, Role: LEADER
I20260807 11:28:20.305404 5820 sys_catalog.cc:564] T 00000000000000000000000000000000 P a706aef041b44cfcb97fb4b4a75b7cb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:28:20.305323 5825 consensus_queue.cc:237] T 00000000000000000000000000000000 P a706aef041b44cfcb97fb4b4a75b7cb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a706aef041b44cfcb97fb4b4a75b7cb6" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 40977 } }
I20260807 11:28:20.308369 5828 sys_catalog.cc:455] T 00000000000000000000000000000000 P a706aef041b44cfcb97fb4b4a75b7cb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a706aef041b44cfcb97fb4b4a75b7cb6. Latest consensus state: current_term: 1 leader_uuid: "a706aef041b44cfcb97fb4b4a75b7cb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a706aef041b44cfcb97fb4b4a75b7cb6" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 40977 } } }
I20260807 11:28:20.308439 5827 sys_catalog.cc:455] T 00000000000000000000000000000000 P a706aef041b44cfcb97fb4b4a75b7cb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a706aef041b44cfcb97fb4b4a75b7cb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a706aef041b44cfcb97fb4b4a75b7cb6" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 40977 } } }
I20260807 11:28:20.308841 5827 sys_catalog.cc:458] T 00000000000000000000000000000000 P a706aef041b44cfcb97fb4b4a75b7cb6 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:20.308845 5828 sys_catalog.cc:458] T 00000000000000000000000000000000 P a706aef041b44cfcb97fb4b4a75b7cb6 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:20.310604 5835 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:28:20.315685 5835 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:28:20.323649 5835 catalog_manager.cc:1347] Generated new cluster ID: 72680dfeca8748cead0dc3d37bb6bcf0
I20260807 11:28:20.323757 5835 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:28:20.331895 5835 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:28:20.333261 5835 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:28:20.341671 5835 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a706aef041b44cfcb97fb4b4a75b7cb6: Generated new TSK 0
I20260807 11:28:20.342486 5835 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:28:20.357308 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.1:0
--local_ip_for_outbound_sockets=127.16.197.1
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:40977
--builtin_ntp_servers=127.16.197.20:33829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 11:28:20.700193 5846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:20.700505 5846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:20.700688 5846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:20.707968 5846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:20.708098 5846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:20.708268 5846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.1
I20260807 11:28:20.717221 5846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:33829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.197.1
--webserver_port=0
--tserver_master_addrs=127.16.197.62:40977
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.1
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:20.718508 5846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:20.720299 5846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:20.735340 5851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:20.735406 5852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:20.735448 5854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:20.736835 5846 server_base.cc:1034] running on GCE node
I20260807 11:28:20.737438 5846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:20.738306 5846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:20.739526 5846 hybrid_clock.cc:648] HybridClock initialized: now 1786102100739487 us; error 50 us; skew 500 ppm
I20260807 11:28:20.739907 5846 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 11:28:20.742059 5846 webserver.cc:460] Webserver started at http://127.16.197.1:44945/ using document root <none> and password file <none>
I20260807 11:28:20.742741 5846 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:20.742872 5846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:20.743248 5846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:20.746244 5846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-0/data/instance:
uuid: "ab976fb1dcb04ec0936eb6b83a4248ce"
format_stamp: "Formatted at 2026-08-07 11:28:20 on dist-test-slave-2w3w"
server_key: "fdd080a3b875565dc3e4edaa997952bb"
server_key_iv: "28a3c4870f40521d905a69414456e6cb"
server_key_version: "encryptionkey@0"
I20260807 11:28:20.747138 5846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance:
uuid: "ab976fb1dcb04ec0936eb6b83a4248ce"
format_stamp: "Formatted at 2026-08-07 11:28:20 on dist-test-slave-2w3w"
server_key: "fdd080a3b875565dc3e4edaa997952bb"
server_key_iv: "28a3c4870f40521d905a69414456e6cb"
server_key_version: "encryptionkey@0"
I20260807 11:28:20.752267 5846 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260807 11:28:20.755707 5860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:20.757015 5846 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 11:28:20.757190 5846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-0/wal
uuid: "ab976fb1dcb04ec0936eb6b83a4248ce"
format_stamp: "Formatted at 2026-08-07 11:28:20 on dist-test-slave-2w3w"
server_key: "fdd080a3b875565dc3e4edaa997952bb"
server_key_iv: "28a3c4870f40521d905a69414456e6cb"
server_key_version: "encryptionkey@0"
I20260807 11:28:20.757411 5846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:20.777192 5846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:20.778059 5846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:20.778384 5846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:20.779417 5846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:28:20.780849 5846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:28:20.780983 5846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:20.781113 5846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:28:20.781204 5846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:20.815281 5846 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.1:36755
I20260807 11:28:20.815330 5977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.1:36755 every 8 connection(s)
I20260807 11:28:20.816697 5846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-0/data/info.pb
I20260807 11:28:20.818753 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 5846
I20260807 11:28:20.818951 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-0/wal/instance
I20260807 11:28:20.819581 17172 external_mini_cluster.cc:1428] Setting key d7faaa89925f7c77e9cec780b3537891
I20260807 11:28:20.822530 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.2:0
--local_ip_for_outbound_sockets=127.16.197.2
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:40977
--builtin_ntp_servers=127.16.197.20:33829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 11:28:20.829876 5980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:20.819428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.1:36139 (local address 127.16.197.62:40977)
0807 11:28:20.819921 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:20.819927 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:20.821364 (+ 1437us) server_negotiation.cc:407] Connection header received
0807 11:28:20.823343 (+ 1979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:20.823349 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:20.823444 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:20.823563 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:20.824655 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:20.825380 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:20.826940 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:20.827406 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:20.827550 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:20.828982 (+ 1432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:20.828987 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:20.828990 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:20.829009 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:20.829049 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:20.829060 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:20.829159 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:20.829396 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 11:28:20.829502 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":234,"threads_started":1}
I20260807 11:28:20.831178 5981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:20.820030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:40977 (local address 127.16.197.1:36139)
0807 11:28:20.821214 (+ 1184us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:20.821250 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:20.823086 (+ 1836us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:20.823793 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:20.823800 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:20.823889 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:28:20.824484 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:20.824501 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:20.825546 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:20.825551 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:20.826788 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:20.826800 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:20.827658 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:20.827666 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:20.827896 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:20.828626 (+ 730us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 11:28:20.828629 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:28:20.828668 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 11:28:20.828764 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:28:20.829161 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:28:20.829176 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:28:20.829185 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 11:28:20.829386 (+ 201us) client_negotiation.cc:240] Negotiation successful
0807 11:28:20.829791 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":364,"threads_started":1}
I20260807 11:28:20.832878 5978 heartbeater.cc:347] Connected to a master server at 127.16.197.62:40977
I20260807 11:28:20.833196 5978 heartbeater.cc:464] Registering TS with master...
I20260807 11:28:20.834059 5978 heartbeater.cc:511] Master 127.16.197.62:40977 requested a full tablet report, sending...
I20260807 11:28:20.836151 5785 ts_manager.cc:194] Registered new tserver with Master: ab976fb1dcb04ec0936eb6b83a4248ce (127.16.197.1:36755)
I20260807 11:28:20.837669 5785 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.197.1:36139
W20260807 11:28:21.177139 5982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:21.177481 5982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:21.177655 5982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:21.184952 5982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:21.185092 5982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:21.185276 5982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.2
I20260807 11:28:21.194561 5982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:33829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.197.2
--webserver_port=0
--tserver_master_addrs=127.16.197.62:40977
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.2
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:21.195935 5982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:21.197727 5982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:21.211863 5990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:21.211901 5988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:21.211935 5987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:21.212689 5982 server_base.cc:1034] running on GCE node
I20260807 11:28:21.213330 5982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:21.214254 5982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:21.215476 5982 hybrid_clock.cc:648] HybridClock initialized: now 1786102101215422 us; error 61 us; skew 500 ppm
I20260807 11:28:21.215780 5982 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 11:28:21.218019 5982 webserver.cc:460] Webserver started at http://127.16.197.2:33567/ using document root <none> and password file <none>
I20260807 11:28:21.218689 5982 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:21.218822 5982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:21.219209 5982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:21.222148 5982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-1/data/instance:
uuid: "bf94ce73454347ebbf92bbcc46b16527"
format_stamp: "Formatted at 2026-08-07 11:28:21 on dist-test-slave-2w3w"
server_key: "267b0fa4adb9cfdfffb151b57a046dfd"
server_key_iv: "115950bf8b1a536165d6262ca39decfa"
server_key_version: "encryptionkey@0"
I20260807 11:28:21.223104 5982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance:
uuid: "bf94ce73454347ebbf92bbcc46b16527"
format_stamp: "Formatted at 2026-08-07 11:28:21 on dist-test-slave-2w3w"
server_key: "267b0fa4adb9cfdfffb151b57a046dfd"
server_key_iv: "115950bf8b1a536165d6262ca39decfa"
server_key_version: "encryptionkey@0"
I20260807 11:28:21.228229 5982 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 11:28:21.231622 5996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:21.232906 5982 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 11:28:21.233098 5982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-1/wal
uuid: "bf94ce73454347ebbf92bbcc46b16527"
format_stamp: "Formatted at 2026-08-07 11:28:21 on dist-test-slave-2w3w"
server_key: "267b0fa4adb9cfdfffb151b57a046dfd"
server_key_iv: "115950bf8b1a536165d6262ca39decfa"
server_key_version: "encryptionkey@0"
I20260807 11:28:21.233310 5982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:21.252344 5982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:21.253171 5982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:21.253476 5982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:21.254534 5982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:28:21.256080 5982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:28:21.256207 5982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:21.256349 5982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:28:21.256441 5982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:21.289656 5982 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.2:32875
I20260807 11:28:21.289706 6114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.2:32875 every 8 connection(s)
I20260807 11:28:21.291298 5982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-1/data/info.pb
I20260807 11:28:21.293859 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 5982
I20260807 11:28:21.294066 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-1/wal/instance
I20260807 11:28:21.294516 17172 external_mini_cluster.cc:1428] Setting key 0c51258e8793e5f5d59b7b9f502e47d7
I20260807 11:28:21.297762 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.197.3:0
--local_ip_for_outbound_sockets=127.16.197.3
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:40977
--builtin_ntp_servers=127.16.197.20:33829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 11:28:21.305959 5980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:21.293902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:33331 (local address 127.16.197.62:40977)
0807 11:28:21.294913 (+ 1011us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:21.294917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:21.295744 (+ 827us) server_negotiation.cc:407] Connection header received
0807 11:28:21.298150 (+ 2406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:21.298154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:21.298222 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:21.298308 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:21.299804 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.300526 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.303049 (+ 2523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.303432 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.303528 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:21.305188 (+ 1660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:21.305192 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:21.305196 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:21.305215 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:21.305245 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:21.305257 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:21.305350 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:21.305634 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 11:28:21.305712 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":887}
I20260807 11:28:21.307446 6117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:21.294468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:40977 (local address 127.16.197.2:33331)
0807 11:28:21.295566 (+ 1098us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:21.295615 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:21.297790 (+ 2175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:21.298585 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:21.298599 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:21.298730 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:28:21.299609 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:21.299637 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.300679 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.300687 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:21.302843 (+ 2156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:21.302861 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.303664 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.303673 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:21.304011 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:21.304791 (+ 780us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 11:28:21.304796 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:28:21.304857 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 11:28:21.304970 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:28:21.305370 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:28:21.305384 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:28:21.305400 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 11:28:21.305595 (+ 195us) client_negotiation.cc:240] Negotiation successful
0807 11:28:21.306026 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":420,"threads_started":1}
I20260807 11:28:21.308925 6115 heartbeater.cc:347] Connected to a master server at 127.16.197.62:40977
I20260807 11:28:21.309268 6115 heartbeater.cc:464] Registering TS with master...
I20260807 11:28:21.310067 6115 heartbeater.cc:511] Master 127.16.197.62:40977 requested a full tablet report, sending...
I20260807 11:28:21.311589 5785 ts_manager.cc:194] Registered new tserver with Master: bf94ce73454347ebbf92bbcc46b16527 (127.16.197.2:32875)
I20260807 11:28:21.312829 5785 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.197.2:33331
W20260807 11:28:21.635515 6118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:21.635828 6118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:21.635919 6118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:21.643334 6118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:21.643477 6118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:21.643658 6118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.197.3
I20260807 11:28:21.652669 6118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:33829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.197.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.197.3
--webserver_port=0
--tserver_master_addrs=127.16.197.62:40977
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.197.3
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:21.654029 6118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:21.656033 6118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:21.669939 6123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:21.670053 6126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:21.670095 6124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:21.670804 6118 server_base.cc:1034] running on GCE node
I20260807 11:28:21.671463 6118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:21.672427 6118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:21.673640 6118 hybrid_clock.cc:648] HybridClock initialized: now 1786102101673583 us; error 55 us; skew 500 ppm
I20260807 11:28:21.673982 6118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 11:28:21.676327 6118 webserver.cc:460] Webserver started at http://127.16.197.3:37487/ using document root <none> and password file <none>
I20260807 11:28:21.677032 6118 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:21.677157 6118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:21.677510 6118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:21.680516 6118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-2/data/instance:
uuid: "1d5b846ebe114b84abbaa1d702f35a99"
format_stamp: "Formatted at 2026-08-07 11:28:21 on dist-test-slave-2w3w"
server_key: "6191751a032e38f59bb5fea71aedb433"
server_key_iv: "d890e6b6e12e131c45c2c4d5a6640ce9"
server_key_version: "encryptionkey@0"
I20260807 11:28:21.681337 6118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance:
uuid: "1d5b846ebe114b84abbaa1d702f35a99"
format_stamp: "Formatted at 2026-08-07 11:28:21 on dist-test-slave-2w3w"
server_key: "6191751a032e38f59bb5fea71aedb433"
server_key_iv: "d890e6b6e12e131c45c2c4d5a6640ce9"
server_key_version: "encryptionkey@0"
I20260807 11:28:21.686555 6118 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 11:28:21.690105 6133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:21.691599 6118 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:28:21.691795 6118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-2/wal
uuid: "1d5b846ebe114b84abbaa1d702f35a99"
format_stamp: "Formatted at 2026-08-07 11:28:21 on dist-test-slave-2w3w"
server_key: "6191751a032e38f59bb5fea71aedb433"
server_key_iv: "d890e6b6e12e131c45c2c4d5a6640ce9"
server_key_version: "encryptionkey@0"
I20260807 11:28:21.692037 6118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:21.717447 6118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:21.718346 6118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:21.718669 6118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:21.719775 6118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 11:28:21.721285 6118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 11:28:21.721395 6118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:21.721544 6118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 11:28:21.721632 6118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:21.755981 6118 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.3:45209
I20260807 11:28:21.756043 6247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.3:45209 every 8 connection(s)
I20260807 11:28:21.757349 6118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-2/data/info.pb
I20260807 11:28:21.759693 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 6118
I20260807 11:28:21.759898 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-2/wal/instance
I20260807 11:28:21.760428 17172 external_mini_cluster.cc:1428] Setting key 4bbb5f30290412dfb19fd48d30c79e19
I20260807 11:28:21.770380 5980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:21.759987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.3:57529 (local address 127.16.197.62:40977)
0807 11:28:21.760212 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:21.760219 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:21.761817 (+ 1598us) server_negotiation.cc:407] Connection header received
0807 11:28:21.764049 (+ 2232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:21.764054 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:21.764139 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:21.764261 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:21.765328 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.766080 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.767682 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.768073 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.768209 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:21.769554 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:21.769562 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:21.769571 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:21.769588 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:21.769619 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:21.769630 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:21.769736 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:21.769948 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 11:28:21.770056 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 11:28:21.771468 6250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:21.760609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:40977 (local address 127.16.197.3:57529)
0807 11:28:21.761666 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:21.761713 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:21.763804 (+ 2091us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:21.764510 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:21.764518 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:21.764608 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:28:21.765153 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:21.765169 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.766251 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.766255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:21.767534 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:21.767546 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.768327 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.768335 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:21.768606 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:21.769251 (+ 645us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 11:28:21.769254 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:28:21.769291 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 11:28:21.769368 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:28:21.769726 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:28:21.769738 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:28:21.769747 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 11:28:21.769937 (+ 190us) client_negotiation.cc:240] Negotiation successful
0807 11:28:21.770328 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":290,"threads_started":1}
I20260807 11:28:21.772825 6248 heartbeater.cc:347] Connected to a master server at 127.16.197.62:40977
I20260807 11:28:21.773135 6248 heartbeater.cc:464] Registering TS with master...
I20260807 11:28:21.773834 6248 heartbeater.cc:511] Master 127.16.197.62:40977 requested a full tablet report, sending...
I20260807 11:28:21.775486 5785 ts_manager.cc:194] Registered new tserver with Master: 1d5b846ebe114b84abbaa1d702f35a99 (127.16.197.3:45209)
I20260807 11:28:21.776382 5785 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.197.3:57529
I20260807 11:28:21.788547 17172 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 11:28:21.798452 5980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:21.792822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59992 (local address 127.16.197.62:40977)
0807 11:28:21.793031 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:21.793036 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:21.793442 (+ 406us) server_negotiation.cc:407] Connection header received
0807 11:28:21.793581 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:21.793585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:21.793650 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:21.793737 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:21.794199 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.794912 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.796343 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.796815 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.796910 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:21.797875 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:21.797880 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:21.797883 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:21.797901 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:21.797930 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:21.797941 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:21.798010 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:21.798135 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 11:28:21.798206 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 11:28:21.802241 5785 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59992:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 11:28:21.809378 5785 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 11:28:21.836664 6264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:21.821625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59646 (local address 127.16.197.3:45209)
0807 11:28:21.822900 (+ 1275us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:21.822908 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:21.827787 (+ 4879us) server_negotiation.cc:407] Connection header received
0807 11:28:21.828197 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:21.828209 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:21.828506 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:21.828743 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:21.829567 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.831260 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.834147 (+ 2887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.835407 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.835551 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:21.835835 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:21.836117 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 11:28:21.836279 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1100,"thread_start_us":926,"threads_started":1}
I20260807 11:28:21.836710 6261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:21.821400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:45209 (local address 127.0.0.1:59646)
0807 11:28:21.827607 (+ 6207us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:21.827657 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:21.827889 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:21.828772 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:21.828780 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:21.828838 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:21.829313 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:21.829331 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.831395 (+ 2064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.831402 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:21.833931 (+ 2529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:21.833946 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.835619 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.835626 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:21.835873 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:21.835932 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 11:28:21.836067 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 11:28:21.836261 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5862,"mutex_wait_us":1,"thread_start_us":5667,"threads_started":1}
I20260807 11:28:21.839097 6262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:21.821483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.2:32875 (local address 127.0.0.1:38568)
0807 11:28:21.830684 (+ 9201us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:21.830704 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:21.830872 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:21.831525 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:21.831535 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:21.831567 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:21.832086 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:21.832101 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.833861 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.833872 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:21.836221 (+ 2349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:21.836236 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.838318 (+ 2082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.838325 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:21.838489 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:21.838519 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 11:28:21.838580 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 11:28:21.838703 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9032,"mutex_wait_us":49,"thread_start_us":8803,"threads_started":1}
I20260807 11:28:21.839586 6269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:21.829050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:36755 (local address 127.0.0.1:45072)
0807 11:28:21.830806 (+ 1756us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:21.830823 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:21.830937 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:21.831780 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:21.831783 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:21.831802 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:21.832387 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:21.832404 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.834323 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.834333 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:21.836491 (+ 2158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:21.836508 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.838798 (+ 2290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.838804 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:21.838990 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:21.839028 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 11:28:21.839129 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 11:28:21.839259 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1638,"mutex_wait_us":3,"thread_start_us":1477,"threads_started":1}
I20260807 11:28:21.839938 6265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:21.822103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38568 (local address 127.16.197.2:32875)
0807 11:28:21.823379 (+ 1276us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:21.823398 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:21.830829 (+ 7431us) server_negotiation.cc:407] Connection header received
0807 11:28:21.831118 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:21.831126 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:21.831333 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:21.831528 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:21.832300 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.833700 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.836459 (+ 2759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.838108 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.839106 (+ 998us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:21.839317 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:21.839403 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 11:28:21.839551 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1115,"thread_start_us":994,"threads_started":1}
I20260807 11:28:21.840747 5978 heartbeater.cc:503] Master 127.16.197.62:40977 was elected leader, sending a full tablet report...
I20260807 11:28:21.841251 6267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:21.823873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45072 (local address 127.16.197.1:36755)
0807 11:28:21.828775 (+ 4902us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:21.828783 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:21.831056 (+ 2273us) server_negotiation.cc:407] Connection header received
0807 11:28:21.831167 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:21.831173 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:21.831333 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:21.831528 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:21.832600 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.834199 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.836715 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.838292 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.838432 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:21.838682 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:21.839966 (+ 1284us) server_negotiation.cc:299] Negotiation successful
0807 11:28:21.840957 (+ 991us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4731,"thread_start_us":4621,"threads_started":1}
I20260807 11:28:21.840690 6182 tablet_service.cc:1467] Processing CreateTablet for tablet 6340441e86f04ae1b5b443bdb3b07b5c (DEFAULT_TABLE table=test-table [id=bee32c56f24b41aa993bc4e93a988bc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:28:21.842264 6182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6340441e86f04ae1b5b443bdb3b07b5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:21.843452 6046 tablet_service.cc:1467] Processing CreateTablet for tablet 6340441e86f04ae1b5b443bdb3b07b5c (DEFAULT_TABLE table=test-table [id=bee32c56f24b41aa993bc4e93a988bc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:28:21.844775 6046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6340441e86f04ae1b5b443bdb3b07b5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:21.845902 5910 tablet_service.cc:1467] Processing CreateTablet for tablet 6340441e86f04ae1b5b443bdb3b07b5c (DEFAULT_TABLE table=test-table [id=bee32c56f24b41aa993bc4e93a988bc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 11:28:21.846923 5910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6340441e86f04ae1b5b443bdb3b07b5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:21.857686 6270 tablet_bootstrap.cc:492] T 6340441e86f04ae1b5b443bdb3b07b5c P ab976fb1dcb04ec0936eb6b83a4248ce: Bootstrap starting.
I20260807 11:28:21.859717 6271 tablet_bootstrap.cc:492] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527: Bootstrap starting.
I20260807 11:28:21.860769 6272 tablet_bootstrap.cc:492] T 6340441e86f04ae1b5b443bdb3b07b5c P 1d5b846ebe114b84abbaa1d702f35a99: Bootstrap starting.
I20260807 11:28:21.862021 6270 tablet_bootstrap.cc:654] T 6340441e86f04ae1b5b443bdb3b07b5c P ab976fb1dcb04ec0936eb6b83a4248ce: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:21.863466 6270 log.cc:824] T 6340441e86f04ae1b5b443bdb3b07b5c P ab976fb1dcb04ec0936eb6b83a4248ce: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:21.864054 6271 tablet_bootstrap.cc:654] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:21.864101 6272 tablet_bootstrap.cc:654] T 6340441e86f04ae1b5b443bdb3b07b5c P 1d5b846ebe114b84abbaa1d702f35a99: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:21.865334 6272 log.cc:824] T 6340441e86f04ae1b5b443bdb3b07b5c P 1d5b846ebe114b84abbaa1d702f35a99: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:21.865525 6271 log.cc:824] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:21.868649 6271 tablet_bootstrap.cc:492] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527: No bootstrap required, opened a new log
I20260807 11:28:21.868728 6272 tablet_bootstrap.cc:492] T 6340441e86f04ae1b5b443bdb3b07b5c P 1d5b846ebe114b84abbaa1d702f35a99: No bootstrap required, opened a new log
I20260807 11:28:21.868973 6271 ts_tablet_manager.cc:1397] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 11:28:21.868994 6272 ts_tablet_manager.cc:1397] T 6340441e86f04ae1b5b443bdb3b07b5c P 1d5b846ebe114b84abbaa1d702f35a99: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 11:28:21.869444 6270 tablet_bootstrap.cc:492] T 6340441e86f04ae1b5b443bdb3b07b5c P ab976fb1dcb04ec0936eb6b83a4248ce: No bootstrap required, opened a new log
I20260807 11:28:21.869716 6270 ts_tablet_manager.cc:1397] T 6340441e86f04ae1b5b443bdb3b07b5c P ab976fb1dcb04ec0936eb6b83a4248ce: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260807 11:28:21.876735 6271 raft_consensus.cc:348] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5b846ebe114b84abbaa1d702f35a99" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45209 } } peers { permanent_uuid: "bf94ce73454347ebbf92bbcc46b16527" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 32875 } } peers { permanent_uuid: "ab976fb1dcb04ec0936eb6b83a4248ce" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 36755 } }
I20260807 11:28:21.877171 6271 raft_consensus.cc:374] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:21.877264 6271 raft_consensus.cc:729] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf94ce73454347ebbf92bbcc46b16527, State: Initialized, Role: FOLLOWER
I20260807 11:28:21.877928 6271 consensus_queue.cc:260] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5b846ebe114b84abbaa1d702f35a99" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45209 } } peers { permanent_uuid: "bf94ce73454347ebbf92bbcc46b16527" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 32875 } } peers { permanent_uuid: "ab976fb1dcb04ec0936eb6b83a4248ce" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 36755 } }
I20260807 11:28:21.879190 6115 heartbeater.cc:503] Master 127.16.197.62:40977 was elected leader, sending a full tablet report...
I20260807 11:28:21.879904 6271 ts_tablet_manager.cc:1428] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260807 11:28:21.879899 6272 raft_consensus.cc:348] T 6340441e86f04ae1b5b443bdb3b07b5c P 1d5b846ebe114b84abbaa1d702f35a99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5b846ebe114b84abbaa1d702f35a99" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45209 } } peers { permanent_uuid: "bf94ce73454347ebbf92bbcc46b16527" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 32875 } } peers { permanent_uuid: "ab976fb1dcb04ec0936eb6b83a4248ce" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 36755 } }
I20260807 11:28:21.880340 6272 raft_consensus.cc:374] T 6340441e86f04ae1b5b443bdb3b07b5c P 1d5b846ebe114b84abbaa1d702f35a99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:21.880450 6272 raft_consensus.cc:729] T 6340441e86f04ae1b5b443bdb3b07b5c P 1d5b846ebe114b84abbaa1d702f35a99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d5b846ebe114b84abbaa1d702f35a99, State: Initialized, Role: FOLLOWER
I20260807 11:28:21.880355 6270 raft_consensus.cc:348] T 6340441e86f04ae1b5b443bdb3b07b5c P ab976fb1dcb04ec0936eb6b83a4248ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5b846ebe114b84abbaa1d702f35a99" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45209 } } peers { permanent_uuid: "bf94ce73454347ebbf92bbcc46b16527" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 32875 } } peers { permanent_uuid: "ab976fb1dcb04ec0936eb6b83a4248ce" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 36755 } }
I20260807 11:28:21.880780 6270 raft_consensus.cc:374] T 6340441e86f04ae1b5b443bdb3b07b5c P ab976fb1dcb04ec0936eb6b83a4248ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:21.880877 6270 raft_consensus.cc:729] T 6340441e86f04ae1b5b443bdb3b07b5c P ab976fb1dcb04ec0936eb6b83a4248ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab976fb1dcb04ec0936eb6b83a4248ce, State: Initialized, Role: FOLLOWER
I20260807 11:28:21.881080 6272 consensus_queue.cc:260] T 6340441e86f04ae1b5b443bdb3b07b5c P 1d5b846ebe114b84abbaa1d702f35a99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5b846ebe114b84abbaa1d702f35a99" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45209 } } peers { permanent_uuid: "bf94ce73454347ebbf92bbcc46b16527" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 32875 } } peers { permanent_uuid: "ab976fb1dcb04ec0936eb6b83a4248ce" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 36755 } }
I20260807 11:28:21.881464 6270 consensus_queue.cc:260] T 6340441e86f04ae1b5b443bdb3b07b5c P ab976fb1dcb04ec0936eb6b83a4248ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5b846ebe114b84abbaa1d702f35a99" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45209 } } peers { permanent_uuid: "bf94ce73454347ebbf92bbcc46b16527" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 32875 } } peers { permanent_uuid: "ab976fb1dcb04ec0936eb6b83a4248ce" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 36755 } }
I20260807 11:28:21.882328 6248 heartbeater.cc:503] Master 127.16.197.62:40977 was elected leader, sending a full tablet report...
I20260807 11:28:21.882876 6272 ts_tablet_manager.cc:1428] T 6340441e86f04ae1b5b443bdb3b07b5c P 1d5b846ebe114b84abbaa1d702f35a99: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260807 11:28:21.883203 6276 raft_consensus.cc:482] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 11:28:21.883533 6276 raft_consensus.cc:504] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5b846ebe114b84abbaa1d702f35a99" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45209 } } peers { permanent_uuid: "bf94ce73454347ebbf92bbcc46b16527" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 32875 } } peers { permanent_uuid: "ab976fb1dcb04ec0936eb6b83a4248ce" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 36755 } }
I20260807 11:28:21.885550 6276 leader_election.cc:288] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d5b846ebe114b84abbaa1d702f35a99 (127.16.197.3:45209), ab976fb1dcb04ec0936eb6b83a4248ce (127.16.197.1:36755)
I20260807 11:28:21.891795 6270 ts_tablet_manager.cc:1428] T 6340441e86f04ae1b5b443bdb3b07b5c P ab976fb1dcb04ec0936eb6b83a4248ce: Time spent starting tablet: real 0.022s user 0.020s sys 0.001s
I20260807 11:28:21.895371 6264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:21.886099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:39427 (local address 127.16.197.3:45209)
0807 11:28:21.886469 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:21.886479 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:21.887028 (+ 549us) server_negotiation.cc:407] Connection header received
0807 11:28:21.888063 (+ 1035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:21.888068 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:21.888141 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:21.888201 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:21.888974 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.890081 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.892877 (+ 2796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.894165 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.894321 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:21.894399 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:21.894911 (+ 512us) server_negotiation.cc:299] Negotiation successful
0807 11:28:21.895105 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202}
I20260807 11:28:21.895473 6279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:21.886038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.3:45209 (local address 127.16.197.2:39427)
0807 11:28:21.886692 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:21.886711 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:21.887900 (+ 1189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:21.888391 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:21.888395 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:21.888421 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:21.888784 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:21.888799 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.890266 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.890271 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:21.892666 (+ 2395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:21.892686 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.894532 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.894539 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:21.894730 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:21.894765 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 11:28:21.894882 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 11:28:21.895090 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"mutex_wait_us":38,"thread_start_us":1259,"threads_started":1}
I20260807 11:28:21.895637 6267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:21.887227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.197.2:59507 (local address 127.16.197.1:36755)
0807 11:28:21.887481 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:21.887490 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:21.887918 (+ 428us) server_negotiation.cc:407] Connection header received
0807 11:28:21.888456 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:21.888462 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:21.888566 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:21.888657 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 11:28:21.889505 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.890544 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.893252 (+ 2708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.894554 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.894698 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:21.894782 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:21.895233 (+ 451us) server_negotiation.cc:299] Negotiation successful
0807 11:28:21.895359 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 11:28:21.896181 6280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:21.886724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:36755 (local address 127.16.197.2:59507)
0807 11:28:21.887798 (+ 1074us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:21.887827 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 11:28:21.888030 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:28:21.888835 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:28:21.888842 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:28:21.888871 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 11:28:21.889272 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:21.889283 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.890779 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.890784 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:21.893039 (+ 2255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:28:21.893053 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.894847 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.894854 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:28:21.895068 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:21.895092 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 11:28:21.895195 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 11:28:21.895328 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":916,"mutex_wait_us":21,"thread_start_us":742,"threads_started":1}
I20260807 11:28:21.896529 6203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6340441e86f04ae1b5b443bdb3b07b5c" candidate_uuid: "bf94ce73454347ebbf92bbcc46b16527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d5b846ebe114b84abbaa1d702f35a99" is_pre_election: true
I20260807 11:28:21.897152 6203 raft_consensus.cc:2454] T 6340441e86f04ae1b5b443bdb3b07b5c P 1d5b846ebe114b84abbaa1d702f35a99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf94ce73454347ebbf92bbcc46b16527 in term 0.
I20260807 11:28:21.897157 5933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6340441e86f04ae1b5b443bdb3b07b5c" candidate_uuid: "bf94ce73454347ebbf92bbcc46b16527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab976fb1dcb04ec0936eb6b83a4248ce" is_pre_election: true
I20260807 11:28:21.897605 5933 raft_consensus.cc:2454] T 6340441e86f04ae1b5b443bdb3b07b5c P ab976fb1dcb04ec0936eb6b83a4248ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf94ce73454347ebbf92bbcc46b16527 in term 0.
I20260807 11:28:21.897914 5997 leader_election.cc:302] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d5b846ebe114b84abbaa1d702f35a99, bf94ce73454347ebbf92bbcc46b16527; no voters:
I20260807 11:28:21.898310 6276 raft_consensus.cc:2781] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 11:28:21.898488 6276 raft_consensus.cc:482] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 11:28:21.898615 6276 raft_consensus.cc:3037] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:21.900904 6276 raft_consensus.cc:504] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5b846ebe114b84abbaa1d702f35a99" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45209 } } peers { permanent_uuid: "bf94ce73454347ebbf92bbcc46b16527" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 32875 } } peers { permanent_uuid: "ab976fb1dcb04ec0936eb6b83a4248ce" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 36755 } }
I20260807 11:28:21.901500 6276 leader_election.cc:288] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527 [CANDIDATE]: Term 1 election: Requested vote from peers 1d5b846ebe114b84abbaa1d702f35a99 (127.16.197.3:45209), ab976fb1dcb04ec0936eb6b83a4248ce (127.16.197.1:36755)
I20260807 11:28:21.901923 6203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6340441e86f04ae1b5b443bdb3b07b5c" candidate_uuid: "bf94ce73454347ebbf92bbcc46b16527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d5b846ebe114b84abbaa1d702f35a99"
I20260807 11:28:21.902014 5933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6340441e86f04ae1b5b443bdb3b07b5c" candidate_uuid: "bf94ce73454347ebbf92bbcc46b16527" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab976fb1dcb04ec0936eb6b83a4248ce"
I20260807 11:28:21.902150 6203 raft_consensus.cc:3037] T 6340441e86f04ae1b5b443bdb3b07b5c P 1d5b846ebe114b84abbaa1d702f35a99 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:21.902218 5933 raft_consensus.cc:3037] T 6340441e86f04ae1b5b443bdb3b07b5c P ab976fb1dcb04ec0936eb6b83a4248ce [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:21.905515 6203 raft_consensus.cc:2454] T 6340441e86f04ae1b5b443bdb3b07b5c P 1d5b846ebe114b84abbaa1d702f35a99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf94ce73454347ebbf92bbcc46b16527 in term 1.
I20260807 11:28:21.905515 5933 raft_consensus.cc:2454] T 6340441e86f04ae1b5b443bdb3b07b5c P ab976fb1dcb04ec0936eb6b83a4248ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf94ce73454347ebbf92bbcc46b16527 in term 1.
I20260807 11:28:21.906008 6000 leader_election.cc:302] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab976fb1dcb04ec0936eb6b83a4248ce, bf94ce73454347ebbf92bbcc46b16527; no voters:
I20260807 11:28:21.906347 6276 raft_consensus.cc:2781] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:21.906873 6276 raft_consensus.cc:686] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527 [term 1 LEADER]: Becoming Leader. State: Replica: bf94ce73454347ebbf92bbcc46b16527, State: Running, Role: LEADER
I20260807 11:28:21.907749 6276 consensus_queue.cc:237] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5b846ebe114b84abbaa1d702f35a99" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45209 } } peers { permanent_uuid: "bf94ce73454347ebbf92bbcc46b16527" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 32875 } } peers { permanent_uuid: "ab976fb1dcb04ec0936eb6b83a4248ce" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 36755 } }
I20260807 11:28:21.912109 5784 catalog_manager.cc:5495] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf94ce73454347ebbf92bbcc46b16527 (127.16.197.2). New cstate: current_term: 1 leader_uuid: "bf94ce73454347ebbf92bbcc46b16527" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5b846ebe114b84abbaa1d702f35a99" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf94ce73454347ebbf92bbcc46b16527" member_type: VOTER last_known_addr { host: "127.16.197.2" port: 32875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ab976fb1dcb04ec0936eb6b83a4248ce" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 36755 } health_report { overall_health: UNKNOWN } } }
I20260807 11:28:21.936158 6265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:21.931235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38580 (local address 127.16.197.2:32875)
0807 11:28:21.931403 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:21.931407 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:21.931424 (+ 17us) server_negotiation.cc:407] Connection header received
0807 11:28:21.931509 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:21.931512 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:21.931602 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:21.931730 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:28:21.932031 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.932745 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.934116 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.934536 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.934642 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:21.935184 (+ 542us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:28:21.935674 (+ 490us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:28:21.935783 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:21.935897 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 11:28:21.935959 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 11:28:21.944205 5933 raft_consensus.cc:1261] T 6340441e86f04ae1b5b443bdb3b07b5c P ab976fb1dcb04ec0936eb6b83a4248ce [term 1 FOLLOWER]: Refusing update from remote peer bf94ce73454347ebbf92bbcc46b16527: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:28:21.944200 6203 raft_consensus.cc:1261] T 6340441e86f04ae1b5b443bdb3b07b5c P 1d5b846ebe114b84abbaa1d702f35a99 [term 1 FOLLOWER]: Refusing update from remote peer bf94ce73454347ebbf92bbcc46b16527: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:28:21.945214 6281 consensus_queue.cc:1035] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab976fb1dcb04ec0936eb6b83a4248ce" member_type: VOTER last_known_addr { host: "127.16.197.1" port: 36755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:28:21.945531 6276 consensus_queue.cc:1035] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d5b846ebe114b84abbaa1d702f35a99" member_type: VOTER last_known_addr { host: "127.16.197.3" port: 45209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:28:21.971455 5784 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:28:21.971745 5784 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 11:28:21.975255 5784 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a706aef041b44cfcb97fb4b4a75b7cb6: Sending DeleteTablet for 3 replicas of tablet 6340441e86f04ae1b5b443bdb3b07b5c
I20260807 11:28:21.976425 6182 tablet_service.cc:1514] Processing DeleteTablet for tablet 6340441e86f04ae1b5b443bdb3b07b5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:28:21 UTC) from {username='slave'} at 127.0.0.1:59646
I20260807 11:28:21.976497 5910 tablet_service.cc:1514] Processing DeleteTablet for tablet 6340441e86f04ae1b5b443bdb3b07b5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:28:21 UTC) from {username='slave'} at 127.0.0.1:45072
I20260807 11:28:21.976482 6046 tablet_service.cc:1514] Processing DeleteTablet for tablet 6340441e86f04ae1b5b443bdb3b07b5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 11:28:21 UTC) from {username='slave'} at 127.0.0.1:38568
I20260807 11:28:21.978768 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 5846
I20260807 11:28:21.995252 6298 tablet_replica.cc:330] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527: stopping tablet replica
I20260807 11:28:21.995600 6295 tablet_replica.cc:330] T 6340441e86f04ae1b5b443bdb3b07b5c P 1d5b846ebe114b84abbaa1d702f35a99: stopping tablet replica
I20260807 11:28:21.996073 6295 raft_consensus.cc:2229] T 6340441e86f04ae1b5b443bdb3b07b5c P 1d5b846ebe114b84abbaa1d702f35a99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:28:21.996085 6298 raft_consensus.cc:2229] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527 [term 1 LEADER]: Raft consensus shutting down.
I20260807 11:28:21.996671 6295 raft_consensus.cc:2258] T 6340441e86f04ae1b5b443bdb3b07b5c P 1d5b846ebe114b84abbaa1d702f35a99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:28:21.996999 6298 raft_consensus.cc:2258] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:28:21.997871 6296 tablet_replica.cc:330] T 6340441e86f04ae1b5b443bdb3b07b5c P ab976fb1dcb04ec0936eb6b83a4248ce: stopping tablet replica
I20260807 11:28:21.998325 6296 raft_consensus.cc:2229] T 6340441e86f04ae1b5b443bdb3b07b5c P ab976fb1dcb04ec0936eb6b83a4248ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 11:28:21.998857 6296 raft_consensus.cc:2258] T 6340441e86f04ae1b5b443bdb3b07b5c P ab976fb1dcb04ec0936eb6b83a4248ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 11:28:21.999507 6295 ts_tablet_manager.cc:1905] T 6340441e86f04ae1b5b443bdb3b07b5c P 1d5b846ebe114b84abbaa1d702f35a99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:28:22.000166 6267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:21.980396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45088 (local address 127.16.197.1:36755)
0807 11:28:21.980578 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:21.980585 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:21.985013 (+ 4428us) server_negotiation.cc:407] Connection header received
0807 11:28:21.986122 (+ 1109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:21.986131 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:21.986304 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:21.986399 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:21.988034 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.989644 (+ 1610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.995082 (+ 5438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:21.995791 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:21.995950 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:21.997895 (+ 1945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:21.997907 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:21.997927 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:21.998770 (+ 843us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:21.998845 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:21.998875 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:21.999048 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:21.999664 (+ 616us) server_negotiation.cc:299] Negotiation successful
0807 11:28:21.999785 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 11:28:22.001019 5973 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:22.001181 6298 ts_tablet_manager.cc:1905] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 11:28:22.009238 6298 ts_tablet_manager.cc:1918] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:28:22.009456 6295 ts_tablet_manager.cc:1918] T 6340441e86f04ae1b5b443bdb3b07b5c P 1d5b846ebe114b84abbaa1d702f35a99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 11:28:22.009580 6298 log.cc:1196] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-1/wal/wals/6340441e86f04ae1b5b443bdb3b07b5c
I20260807 11:28:22.009702 6295 log.cc:1196] T 6340441e86f04ae1b5b443bdb3b07b5c P 1d5b846ebe114b84abbaa1d702f35a99: Deleting WAL directory at /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786101765888975-17172-0/minicluster-data/ts-2/wal/wals/6340441e86f04ae1b5b443bdb3b07b5c
I20260807 11:28:22.010394 6298 ts_tablet_manager.cc:1939] T 6340441e86f04ae1b5b443bdb3b07b5c P bf94ce73454347ebbf92bbcc46b16527: Deleting consensus metadata
I20260807 11:28:22.010492 6295 ts_tablet_manager.cc:1939] T 6340441e86f04ae1b5b443bdb3b07b5c P 1d5b846ebe114b84abbaa1d702f35a99: Deleting consensus metadata
I20260807 11:28:22.012528 5770 catalog_manager.cc:4841] TS bf94ce73454347ebbf92bbcc46b16527 (127.16.197.2:32875): tablet 6340441e86f04ae1b5b443bdb3b07b5c (table test-table [id=bee32c56f24b41aa993bc4e93a988bc0]) successfully deleted
I20260807 11:28:22.012697 5768 catalog_manager.cc:4841] TS 1d5b846ebe114b84abbaa1d702f35a99 (127.16.197.3:45209): tablet 6340441e86f04ae1b5b443bdb3b07b5c (table test-table [id=bee32c56f24b41aa993bc4e93a988bc0]) successfully deleted
I20260807 11:28:22.105067 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 5846
I20260807 11:28:22.126230 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 5982
W20260807 11:28:22.126390 5771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.197.1:36755 (error 108)
I20260807 11:28:22.129448 6269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:22.128951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:36755 (local address 127.0.0.1:45102)
0807 11:28:22.129136 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:22.129263 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260807 11:28:22.129763 5771 catalog_manager.cc:4584] TS ab976fb1dcb04ec0936eb6b83a4248ce (127.16.197.1:36755): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6340441e86f04ae1b5b443bdb3b07b5c: Network error: Client connection negotiation failed: client connection to 127.16.197.1:36755: connect: Connection refused (error 111)
I20260807 11:28:22.134505 6265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:22.127239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38584 (local address 127.16.197.2:32875)
0807 11:28:22.127462 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:22.127468 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:22.127489 (+ 21us) server_negotiation.cc:407] Connection header received
0807 11:28:22.127559 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:22.127562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:22.127652 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:22.127771 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:22.128331 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:22.129426 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:22.130937 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:22.131523 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:22.131674 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:22.132842 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:22.132855 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:22.132869 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:22.133590 (+ 721us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:22.133644 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:22.133664 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:22.133781 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:22.133950 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 11:28:22.134073 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 11:28:22.135416 6110 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:22.183890 6269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:22.183326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:36755 (local address 127.0.0.1:45116)
0807 11:28:22.183558 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:22.183687 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 11:28:22.184942 6269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:22.184548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:36755 (local address 127.0.0.1:45126)
0807 11:28:22.184698 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:22.184783 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 11:28:22.245656 6269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:22.245048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:36755 (local address 127.0.0.1:45134)
0807 11:28:22.245322 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:22.245440 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 11:28:22.246804 6269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:22.246391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:36755 (local address 127.0.0.1:45144)
0807 11:28:22.246559 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:22.246648 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 11:28:22.252230 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 5982
I20260807 11:28:22.273687 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 6118
I20260807 11:28:22.284570 6264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:22.274511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59652 (local address 127.16.197.3:45209)
0807 11:28:22.274739 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:22.274746 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:22.274769 (+ 23us) server_negotiation.cc:407] Connection header received
0807 11:28:22.274935 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:22.274941 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:22.275073 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:22.275180 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:22.275608 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:22.276775 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:22.278084 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:22.278731 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:22.278846 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:22.283249 (+ 4403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:22.283259 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:22.283274 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:22.283850 (+ 576us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:22.283901 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:22.283921 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:22.284018 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:22.284127 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 11:28:22.284219 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 11:28:22.285254 6243 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:22.327395 6269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:22.326777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:36755 (local address 127.0.0.1:45148)
0807 11:28:22.327045 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:22.327186 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260807 11:28:22.328522 6269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:22.328087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:36755 (local address 127.0.0.1:45156)
0807 11:28:22.328308 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:22.328392 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260807 11:28:22.375936 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 6118
I20260807 11:28:22.395610 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 5750
I20260807 11:28:22.396744 5815 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:22.506310 6269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:22.505613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.1:36755 (local address 127.0.0.1:45168)
0807 11:28:22.505837 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 11:28:22.505957 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.1:36755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 11:28:22.506569 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 5750
2026-08-07T11:28:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2761 ms)
[----------] 6 tests from EncryptionPolicyTest (22258 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:28:22 dist-test-slave-2w3w krb5kdc[6307](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:28:22 dist-test-slave-2w3w krb5kdc[6307](info): set up 2 sockets
Aug 07 11:28:22 dist-test-slave-2w3w krb5kdc[6307](info): commencing operation
krb5kdc: starting...
W20260807 11:28:23.669505 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.002s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:28:23 dist-test-slave-2w3w krb5kdc[6307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:28:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:28:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 11:28:23.971211 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36625
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:24.315507 6320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:24.315920 6320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:24.316116 6320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:24.323540 6320 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:28:24.323693 6320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:24.323799 6320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:24.323926 6320 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:28:24.324002 6320 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:28:24.333976 6320 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36625
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:24.335347 6320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:24.336678 6320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:24.349166 6325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:24.349134 6329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:24.349092 6327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:24.349929 6320 server_base.cc:1034] running on GCE node
I20260807 11:28:24.350826 6320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:24.352403 6320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:24.353617 6320 hybrid_clock.cc:648] HybridClock initialized: now 1786102104353555 us; error 53 us; skew 500 ppm
I20260807 11:28:24.354074 6320 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:24 dist-test-slave-2w3w krb5kdc[6307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102104, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:24.357589 6320 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 11:28:24.359110 6320 webserver.cc:460] Webserver started at http://127.0.0.1:41337/ using document root <none> and password file <none>
I20260807 11:28:24.359819 6320 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:24.359958 6320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:24.360283 6320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:24.363476 6320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "b7b80ba4fee64ae18aa460d6a89bf236"
format_stamp: "Formatted at 2026-08-07 11:28:24 on dist-test-slave-2w3w"
server_key: "adbb3ec0eea177f9e1902b58b03be3c2"
server_key_iv: "2474ce3682ed263bc47cb43897c1e794"
server_key_version: "encryptionkey@0"
I20260807 11:28:24.364277 6320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "b7b80ba4fee64ae18aa460d6a89bf236"
format_stamp: "Formatted at 2026-08-07 11:28:24 on dist-test-slave-2w3w"
server_key: "adbb3ec0eea177f9e1902b58b03be3c2"
server_key_iv: "2474ce3682ed263bc47cb43897c1e794"
server_key_version: "encryptionkey@0"
I20260807 11:28:24.369431 6320 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 11:28:24.372836 6336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:24.374120 6320 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 11:28:24.374318 6320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "b7b80ba4fee64ae18aa460d6a89bf236"
format_stamp: "Formatted at 2026-08-07 11:28:24 on dist-test-slave-2w3w"
server_key: "adbb3ec0eea177f9e1902b58b03be3c2"
server_key_iv: "2474ce3682ed263bc47cb43897c1e794"
server_key_version: "encryptionkey@0"
I20260807 11:28:24.374528 6320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:24.396358 6320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:24.400542 6320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:24.400854 6320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:24.419925 6320 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36625
I20260807 11:28:24.419963 6392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36625 every 8 connection(s)
I20260807 11:28:24.421319 6320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:28:24.424998 6393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:24.430610 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 6320
I20260807 11:28:24.430866 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:28:24.431524 17172 external_mini_cluster.cc:1428] Setting key 879114eac48b5dd3cbba01729a11c9e8
I20260807 11:28:24.435815 6393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7b80ba4fee64ae18aa460d6a89bf236: Bootstrap starting.
I20260807 11:28:24.439627 6393 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7b80ba4fee64ae18aa460d6a89bf236: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:24.440771 6393 log.cc:824] T 00000000000000000000000000000000 P b7b80ba4fee64ae18aa460d6a89bf236: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:24.443974 6393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7b80ba4fee64ae18aa460d6a89bf236: No bootstrap required, opened a new log
Aug 07 11:28:24 dist-test-slave-2w3w krb5kdc[6307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 11:28:24.454802 6393 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7b80ba4fee64ae18aa460d6a89bf236 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7b80ba4fee64ae18aa460d6a89bf236" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36625 } }
I20260807 11:28:24.455381 6393 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7b80ba4fee64ae18aa460d6a89bf236 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:24.455528 6393 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7b80ba4fee64ae18aa460d6a89bf236 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7b80ba4fee64ae18aa460d6a89bf236, State: Initialized, Role: FOLLOWER
I20260807 11:28:24.456259 6393 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7b80ba4fee64ae18aa460d6a89bf236 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7b80ba4fee64ae18aa460d6a89bf236" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36625 } }
I20260807 11:28:24.456578 6393 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7b80ba4fee64ae18aa460d6a89bf236 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:28:24.456732 6393 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7b80ba4fee64ae18aa460d6a89bf236 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:28:24.456936 6393 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7b80ba4fee64ae18aa460d6a89bf236 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:24.457208 6396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:24.433620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44315 (local address 127.0.0.1:36625)
0807 11:28:24.434442 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:24.434454 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:24.434496 (+ 42us) server_negotiation.cc:407] Connection header received
0807 11:28:24.436267 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:24.436290 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:24.436667 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:24.436980 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:24.439043 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:24.440456 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:24.442116 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:24.442811 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:24.442949 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:24.448005 (+ 5056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:24.448020 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:24.448040 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:24.448092 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:24.452230 (+ 4138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:24.453053 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:24.453062 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:24.453071 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:24.453287 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:24.453716 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:24.453721 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:24.453724 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:24.454404 (+ 680us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:24.454746 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:24.455157 (+ 411us) server_negotiation.cc:299] Negotiation successful
0807 11:28:24.455628 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":397,"threads_started":1}
I20260807 11:28:24.460188 6393 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7b80ba4fee64ae18aa460d6a89bf236 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7b80ba4fee64ae18aa460d6a89bf236" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36625 } }
I20260807 11:28:24.460774 6393 leader_election.cc:302] T 00000000000000000000000000000000 P b7b80ba4fee64ae18aa460d6a89bf236 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7b80ba4fee64ae18aa460d6a89bf236; no voters:
I20260807 11:28:24.461500 6393 leader_election.cc:288] T 00000000000000000000000000000000 P b7b80ba4fee64ae18aa460d6a89bf236 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:28:24.461618 6398 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7b80ba4fee64ae18aa460d6a89bf236 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:24.461650 17172 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 11:28:24.462795 6398 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7b80ba4fee64ae18aa460d6a89bf236 [term 1 LEADER]: Becoming Leader. State: Replica: b7b80ba4fee64ae18aa460d6a89bf236, State: Running, Role: LEADER
I20260807 11:28:24.463435 6398 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7b80ba4fee64ae18aa460d6a89bf236 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7b80ba4fee64ae18aa460d6a89bf236" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36625 } }
I20260807 11:28:24.463961 6393 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7b80ba4fee64ae18aa460d6a89bf236 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:28:24.475656 6400 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7b80ba4fee64ae18aa460d6a89bf236 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7b80ba4fee64ae18aa460d6a89bf236. Latest consensus state: current_term: 1 leader_uuid: "b7b80ba4fee64ae18aa460d6a89bf236" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7b80ba4fee64ae18aa460d6a89bf236" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36625 } } }
I20260807 11:28:24.476115 6400 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7b80ba4fee64ae18aa460d6a89bf236 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:24.477728 6399 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7b80ba4fee64ae18aa460d6a89bf236 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7b80ba4fee64ae18aa460d6a89bf236" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7b80ba4fee64ae18aa460d6a89bf236" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36625 } } }
I20260807 11:28:24.478269 6399 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7b80ba4fee64ae18aa460d6a89bf236 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:24.479467 6412 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:28:24.486433 6412 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:28:24.490245 6396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:24.467399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50355 (local address 127.0.0.1:36625)
0807 11:28:24.467610 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:24.467620 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:24.471592 (+ 3972us) server_negotiation.cc:407] Connection header received
0807 11:28:24.471812 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:24.471818 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:24.471901 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:24.472023 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:24.473495 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:24.474422 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:24.476692 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:24.477682 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:24.477819 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:24.481394 (+ 3575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:24.481400 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:24.481403 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:24.481437 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:24.485595 (+ 4158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:24.486766 (+ 1171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:24.486771 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:24.486774 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:24.486981 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:24.487653 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:24.487658 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:24.487661 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:24.488112 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:24.489382 (+ 1270us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:24.489572 (+ 190us) server_negotiation.cc:299] Negotiation successful
0807 11:28:24.489768 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 11:28:24.499310 6412 catalog_manager.cc:1347] Generated new cluster ID: 01d3b59828784af8a6b9cb4854be2c83
I20260807 11:28:24.499454 6412 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:28:24.507094 6412 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:28:24.508363 6412 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:28:24.522395 6412 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7b80ba4fee64ae18aa460d6a89bf236: Generated new TSK 0
I20260807 11:28:24.523290 6412 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:28:24.533078 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 6320
I20260807 11:28:24.534914 6387 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:24.626237 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 6320
W20260807 11:28:24.645694 6305 connection.cc:535] client connection to 127.0.0.1:36625 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36625): Connection reset by peer (error 104)
2026-08-07T11:28:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2114 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:28:24 dist-test-slave-2w3w krb5kdc[6427](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:28:24 dist-test-slave-2w3w krb5kdc[6427](info): set up 2 sockets
Aug 07 11:28:24 dist-test-slave-2w3w krb5kdc[6427](info): commencing operation
krb5kdc: starting...
W20260807 11:28:25.764201 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:28:25 dist-test-slave-2w3w krb5kdc[6427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:28:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:28:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 11:28:26.052490 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45275
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:26.391958 6440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:26.392271 6440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:26.392367 6440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:26.399799 6440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:28:26.399926 6440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:26.399976 6440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:26.400040 6440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:28:26.400085 6440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:28:26.410326 6440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45275
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:26.411685 6440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:26.412952 6440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:26.424607 6445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:26.424708 6446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:26.424578 6449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:26.425789 6440 server_base.cc:1034] running on GCE node
I20260807 11:28:26.426502 6440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:26.427917 6440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:26.429116 6440 hybrid_clock.cc:648] HybridClock initialized: now 1786102106429067 us; error 38 us; skew 500 ppm
I20260807 11:28:26.429436 6440 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:26 dist-test-slave-2w3w krb5kdc[6427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102106, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:26.432690 6440 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 11:28:26.434122 6440 webserver.cc:460] Webserver started at http://127.0.0.1:46453/ using document root <none> and password file <none>
I20260807 11:28:26.434782 6440 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:26.434895 6440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:26.435226 6440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:26.438337 6440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "6c4dcc7d6f964dd28062ad60241dd625"
format_stamp: "Formatted at 2026-08-07 11:28:26 on dist-test-slave-2w3w"
server_key: "4e24bf696bd5289966b888bc966339cb"
server_key_iv: "f33fa17aa72d99e67404efce2f6383e4"
server_key_version: "encryptionkey@0"
I20260807 11:28:26.439152 6440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "6c4dcc7d6f964dd28062ad60241dd625"
format_stamp: "Formatted at 2026-08-07 11:28:26 on dist-test-slave-2w3w"
server_key: "4e24bf696bd5289966b888bc966339cb"
server_key_iv: "f33fa17aa72d99e67404efce2f6383e4"
server_key_version: "encryptionkey@0"
I20260807 11:28:26.444434 6440 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260807 11:28:26.447856 6456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:26.449180 6440 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 11:28:26.449369 6440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "6c4dcc7d6f964dd28062ad60241dd625"
format_stamp: "Formatted at 2026-08-07 11:28:26 on dist-test-slave-2w3w"
server_key: "4e24bf696bd5289966b888bc966339cb"
server_key_iv: "f33fa17aa72d99e67404efce2f6383e4"
server_key_version: "encryptionkey@0"
I20260807 11:28:26.449572 6440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:26.460085 6440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:26.464051 6440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:26.464387 6440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:26.482810 6440 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45275
I20260807 11:28:26.482847 6510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45275 every 8 connection(s)
I20260807 11:28:26.484238 6440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:28:26.487906 6511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:26.491501 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 6440
I20260807 11:28:26.491704 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:28:26.492273 17172 external_mini_cluster.cc:1428] Setting key 640e954341ff02b34c92a296bc4913e1
I20260807 11:28:26.500626 6511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c4dcc7d6f964dd28062ad60241dd625: Bootstrap starting.
I20260807 11:28:26.504326 6511 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c4dcc7d6f964dd28062ad60241dd625: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:26.505479 6511 log.cc:824] T 00000000000000000000000000000000 P 6c4dcc7d6f964dd28062ad60241dd625: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:26.505709 6514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:26.494153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45709 (local address 127.0.0.1:45275)
0807 11:28:26.494939 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:26.494950 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:26.495048 (+ 98us) server_negotiation.cc:407] Connection header received
0807 11:28:26.496704 (+ 1656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:26.496726 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:26.497116 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:26.497436 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:26.498018 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:26.499400 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:26.501285 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:26.501925 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:26.502054 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:26.503436 (+ 1382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:26.503444 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:26.503456 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:26.503965 (+ 509us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:26.504036 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:26.504065 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:26.504181 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:26.504549 (+ 368us) server_negotiation.cc:299] Negotiation successful
0807 11:28:26.504688 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":368,"threads_started":1}
I20260807 11:28:26.508378 6511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c4dcc7d6f964dd28062ad60241dd625: No bootstrap required, opened a new log
I20260807 11:28:26.509516 17172 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 11:28:26.516548 6511 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c4dcc7d6f964dd28062ad60241dd625 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4dcc7d6f964dd28062ad60241dd625" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45275 } }
I20260807 11:28:26.516937 6511 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c4dcc7d6f964dd28062ad60241dd625 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:26.517040 6511 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c4dcc7d6f964dd28062ad60241dd625 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c4dcc7d6f964dd28062ad60241dd625, State: Initialized, Role: FOLLOWER
I20260807 11:28:26.517683 6511 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c4dcc7d6f964dd28062ad60241dd625 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4dcc7d6f964dd28062ad60241dd625" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45275 } }
I20260807 11:28:26.517961 6511 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c4dcc7d6f964dd28062ad60241dd625 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:28:26.518114 6511 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c4dcc7d6f964dd28062ad60241dd625 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:28:26.518311 6511 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c4dcc7d6f964dd28062ad60241dd625 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:26.520534 6514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:26.513989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49487 (local address 127.0.0.1:45275)
0807 11:28:26.514152 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:26.514159 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:26.514629 (+ 470us) server_negotiation.cc:407] Connection header received
0807 11:28:26.514845 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:26.514850 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:26.514924 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:26.515057 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:26.515602 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:26.516339 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:26.517835 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:26.518424 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:26.518562 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:26.519759 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:26.519765 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:26.519769 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:26.519796 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:26.519857 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:26.519874 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:26.519997 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:26.520189 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 11:28:26.520243 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 11:28:26.520874 6511 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c4dcc7d6f964dd28062ad60241dd625 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4dcc7d6f964dd28062ad60241dd625" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45275 } }
I20260807 11:28:26.521394 6511 leader_election.cc:302] T 00000000000000000000000000000000 P 6c4dcc7d6f964dd28062ad60241dd625 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c4dcc7d6f964dd28062ad60241dd625; no voters:
I20260807 11:28:26.522053 6511 leader_election.cc:288] T 00000000000000000000000000000000 P 6c4dcc7d6f964dd28062ad60241dd625 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:28:26.522159 6521 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c4dcc7d6f964dd28062ad60241dd625 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:26.523026 6521 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c4dcc7d6f964dd28062ad60241dd625 [term 1 LEADER]: Becoming Leader. State: Replica: 6c4dcc7d6f964dd28062ad60241dd625, State: Running, Role: LEADER
I20260807 11:28:26.523607 6521 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c4dcc7d6f964dd28062ad60241dd625 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4dcc7d6f964dd28062ad60241dd625" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45275 } }
I20260807 11:28:26.524545 6511 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c4dcc7d6f964dd28062ad60241dd625 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:28:26.526506 6522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c4dcc7d6f964dd28062ad60241dd625 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c4dcc7d6f964dd28062ad60241dd625" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4dcc7d6f964dd28062ad60241dd625" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45275 } } }
I20260807 11:28:26.526991 6522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c4dcc7d6f964dd28062ad60241dd625 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:26.527662 6523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c4dcc7d6f964dd28062ad60241dd625 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c4dcc7d6f964dd28062ad60241dd625. Latest consensus state: current_term: 1 leader_uuid: "6c4dcc7d6f964dd28062ad60241dd625" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c4dcc7d6f964dd28062ad60241dd625" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45275 } } }
I20260807 11:28:26.528115 6523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c4dcc7d6f964dd28062ad60241dd625 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:26.529997 6528 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:28:26.536754 6528 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:28:26.548904 6528 catalog_manager.cc:1347] Generated new cluster ID: 7e3fa39090bb46edbde6eca04105e69c
I20260807 11:28:26.549165 6528 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:28:26.582281 6528 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:28:26.583693 6528 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:28:26.596733 6528 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c4dcc7d6f964dd28062ad60241dd625: Generated new TSK 0
I20260807 11:28:26.597637 6528 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:28:26.610137 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 6440
W20260807 11:28:26.611399 6506 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45709
W20260807 11:28:26.612179 17172 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 6440: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 11:28:26.612373 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 6440
W20260807 11:28:26.632052 6425 connection.cc:535] client connection to 127.0.0.1:45275 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45275): Connection reset by peer (error 104)
2026-08-07T11:28:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1986 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:28:26 dist-test-slave-2w3w krb5kdc[6545](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:28:26 dist-test-slave-2w3w krb5kdc[6545](info): set up 2 sockets
Aug 07 11:28:26 dist-test-slave-2w3w krb5kdc[6545](info): commencing operation
krb5kdc: starting...
W20260807 11:28:27.761032 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.000s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:28:27 dist-test-slave-2w3w krb5kdc[6545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:28:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:28:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 11:28:28.058243 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46279
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46279
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:28.409612 6558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:28.409976 6558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:28.410151 6558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:28.417568 6558 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:28:28.417742 6558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:28.417847 6558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:28.417969 6558 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:28:28.418053 6558 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:28:28.428128 6558 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46279
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:28.429508 6558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:28.430877 6558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:28.442867 6563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:28.442911 6564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:28.442975 6566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:28.444243 6558 server_base.cc:1034] running on GCE node
I20260807 11:28:28.445010 6558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:28.446470 6558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:28.447676 6558 hybrid_clock.cc:648] HybridClock initialized: now 1786102108447615 us; error 51 us; skew 500 ppm
I20260807 11:28:28.448040 6558 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:28 dist-test-slave-2w3w krb5kdc[6545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102108, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:28.454654 6558 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 11:28:28.456349 6558 webserver.cc:460] Webserver started at http://127.0.0.1:45501/ using document root <none> and password file <none>
I20260807 11:28:28.457064 6558 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:28.457201 6558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:28.457535 6558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:28.460676 6558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "d9ac8d7180224828bbd885d791fe7807"
format_stamp: "Formatted at 2026-08-07 11:28:28 on dist-test-slave-2w3w"
server_key: "681e1980dd76ba091bfb52b274c97220"
server_key_iv: "23b8a08d56d0a22d7294965c6fad6992"
server_key_version: "encryptionkey@0"
I20260807 11:28:28.461509 6558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "d9ac8d7180224828bbd885d791fe7807"
format_stamp: "Formatted at 2026-08-07 11:28:28 on dist-test-slave-2w3w"
server_key: "681e1980dd76ba091bfb52b274c97220"
server_key_iv: "23b8a08d56d0a22d7294965c6fad6992"
server_key_version: "encryptionkey@0"
I20260807 11:28:28.466995 6558 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 11:28:28.470497 6573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:28.471980 6558 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 11:28:28.472173 6558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "d9ac8d7180224828bbd885d791fe7807"
format_stamp: "Formatted at 2026-08-07 11:28:28 on dist-test-slave-2w3w"
server_key: "681e1980dd76ba091bfb52b274c97220"
server_key_iv: "23b8a08d56d0a22d7294965c6fad6992"
server_key_version: "encryptionkey@0"
I20260807 11:28:28.472384 6558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:28.478791 6558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:28.482893 6558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:28.483268 6558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:28.502581 6558 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46279
I20260807 11:28:28.502631 6627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46279 every 8 connection(s)
I20260807 11:28:28.504032 6558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:28:28.507187 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 6558
I20260807 11:28:28.507403 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:28:28.508008 17172 external_mini_cluster.cc:1428] Setting key 423433aaf75c902331d178985ee3580a
I20260807 11:28:28.508210 6628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:28.516315 6631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:28.509791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46571 (local address 127.0.0.1:46279)
0807 11:28:28.510594 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:28.510605 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:28.510649 (+ 44us) server_negotiation.cc:407] Connection header received
0807 11:28:28.512903 (+ 2254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:28.512925 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:28.513276 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:28.513632 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:28.514729 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:28.514735 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:28.514748 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:28.514762 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:28.514830 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:28.514849 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:28.515084 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:28.515190 (+ 106us) server_negotiation.cc:299] Negotiation successful
0807 11:28:28.515408 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":364,"threads_started":1}
I20260807 11:28:28.520032 17172 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 11:28:28.522217 6628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9ac8d7180224828bbd885d791fe7807: Bootstrap starting.
I20260807 11:28:28.526392 6628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9ac8d7180224828bbd885d791fe7807: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:28.527810 6628 log.cc:824] T 00000000000000000000000000000000 P d9ac8d7180224828bbd885d791fe7807: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:28.528079 6631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:28.525274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40471 (local address 127.0.0.1:46279)
0807 11:28:28.525477 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:28.525486 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:28.525968 (+ 482us) server_negotiation.cc:407] Connection header received
0807 11:28:28.526111 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:28.526118 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:28.526190 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:28.526314 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:28.527268 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:28.527285 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:28.527293 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:28.527337 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:28.527458 (+ 121us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:28.527489 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:28.527618 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:28.527694 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 11:28:28.527751 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 11:28:28.530763 6628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9ac8d7180224828bbd885d791fe7807: No bootstrap required, opened a new log
I20260807 11:28:28.538151 6628 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9ac8d7180224828bbd885d791fe7807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9ac8d7180224828bbd885d791fe7807" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46279 } }
I20260807 11:28:28.538496 6628 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9ac8d7180224828bbd885d791fe7807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:28.538570 6628 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9ac8d7180224828bbd885d791fe7807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9ac8d7180224828bbd885d791fe7807, State: Initialized, Role: FOLLOWER
I20260807 11:28:28.539193 6628 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9ac8d7180224828bbd885d791fe7807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9ac8d7180224828bbd885d791fe7807" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46279 } }
I20260807 11:28:28.539400 6628 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9ac8d7180224828bbd885d791fe7807 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:28:28.539498 6628 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9ac8d7180224828bbd885d791fe7807 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:28:28.539618 6628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9ac8d7180224828bbd885d791fe7807 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:28.541879 6628 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9ac8d7180224828bbd885d791fe7807 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9ac8d7180224828bbd885d791fe7807" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46279 } }
I20260807 11:28:28.542323 6628 leader_election.cc:302] T 00000000000000000000000000000000 P d9ac8d7180224828bbd885d791fe7807 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9ac8d7180224828bbd885d791fe7807; no voters:
I20260807 11:28:28.542870 6628 leader_election.cc:288] T 00000000000000000000000000000000 P d9ac8d7180224828bbd885d791fe7807 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:28:28.543043 6638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9ac8d7180224828bbd885d791fe7807 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:28.543820 6638 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9ac8d7180224828bbd885d791fe7807 [term 1 LEADER]: Becoming Leader. State: Replica: d9ac8d7180224828bbd885d791fe7807, State: Running, Role: LEADER
I20260807 11:28:28.544381 6638 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9ac8d7180224828bbd885d791fe7807 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9ac8d7180224828bbd885d791fe7807" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46279 } }
I20260807 11:28:28.545145 6628 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9ac8d7180224828bbd885d791fe7807 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:28:28.547081 6640 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9ac8d7180224828bbd885d791fe7807 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9ac8d7180224828bbd885d791fe7807. Latest consensus state: current_term: 1 leader_uuid: "d9ac8d7180224828bbd885d791fe7807" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9ac8d7180224828bbd885d791fe7807" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46279 } } }
I20260807 11:28:28.547538 6640 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9ac8d7180224828bbd885d791fe7807 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:28.547554 6639 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9ac8d7180224828bbd885d791fe7807 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9ac8d7180224828bbd885d791fe7807" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9ac8d7180224828bbd885d791fe7807" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46279 } } }
I20260807 11:28:28.547875 6639 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9ac8d7180224828bbd885d791fe7807 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:28.549969 6644 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:28:28.555833 6644 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:28:28.564944 6644 catalog_manager.cc:1347] Generated new cluster ID: 0d1dd56efe4348589a61c7a99759f409
I20260807 11:28:28.565053 6644 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:28:28.583367 6644 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:28:28.583823 6644 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:28:28.591470 6644 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9ac8d7180224828bbd885d791fe7807: Generated new TSK 0
I20260807 11:28:28.592335 6644 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:28:28.598034 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 6558
W20260807 11:28:28.599375 6623 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46571
W20260807 11:28:28.600178 17172 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 6558: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 11:28:28.600373 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 6558
2026-08-07T11:28:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1987 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:28:28 dist-test-slave-2w3w krb5kdc[6659](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:28:28 dist-test-slave-2w3w krb5kdc[6659](info): set up 2 sockets
Aug 07 11:28:28 dist-test-slave-2w3w krb5kdc[6659](info): commencing operation
krb5kdc: starting...
W20260807 11:28:29.762853 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.004s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:28:29 dist-test-slave-2w3w krb5kdc[6659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:28:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:28:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 11:28:30.059381 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41041
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:30.400956 6673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:30.401265 6673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:30.401453 6673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:30.408798 6673 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:28:30.408964 6673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:30.409085 6673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:30.409224 6673 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:28:30.409308 6673 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:28:30.419437 6673 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:30.420822 6673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:30.422196 6673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:30.434283 6679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:30.434216 6683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:30.434232 6681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:30.435554 6673 server_base.cc:1034] running on GCE node
I20260807 11:28:30.436404 6673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:30.437795 6673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:30.439041 6673 hybrid_clock.cc:648] HybridClock initialized: now 1786102110438974 us; error 51 us; skew 500 ppm
I20260807 11:28:30.439366 6673 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:30 dist-test-slave-2w3w krb5kdc[6659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102110, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:30.442585 6673 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 11:28:30.444247 6673 webserver.cc:460] Webserver started at http://127.0.0.1:42787/ using document root <none> and password file <none>
I20260807 11:28:30.444955 6673 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:30.445106 6673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:30.445429 6673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:30.448567 6673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "ca0af031e22045c88a2838a61a15c4c8"
format_stamp: "Formatted at 2026-08-07 11:28:30 on dist-test-slave-2w3w"
server_key: "db9fb6f3c4ca68cffb8978299a13543f"
server_key_iv: "82ef85b031022ed74ee91fc2da22c895"
server_key_version: "encryptionkey@0"
I20260807 11:28:30.449385 6673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "ca0af031e22045c88a2838a61a15c4c8"
format_stamp: "Formatted at 2026-08-07 11:28:30 on dist-test-slave-2w3w"
server_key: "db9fb6f3c4ca68cffb8978299a13543f"
server_key_iv: "82ef85b031022ed74ee91fc2da22c895"
server_key_version: "encryptionkey@0"
I20260807 11:28:30.455351 6673 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 11:28:30.459275 6690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:30.460681 6673 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 11:28:30.460901 6673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "ca0af031e22045c88a2838a61a15c4c8"
format_stamp: "Formatted at 2026-08-07 11:28:30 on dist-test-slave-2w3w"
server_key: "db9fb6f3c4ca68cffb8978299a13543f"
server_key_iv: "82ef85b031022ed74ee91fc2da22c895"
server_key_version: "encryptionkey@0"
I20260807 11:28:30.461123 6673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:30.482497 6673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:30.487438 6673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:30.487766 6673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:30.507015 6673 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41041
I20260807 11:28:30.507082 6742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41041 every 8 connection(s)
I20260807 11:28:30.508473 6673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:28:30.509634 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 6673
I20260807 11:28:30.509819 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:28:30.510416 17172 external_mini_cluster.cc:1428] Setting key f1b59cd9eee042e5d1a35203b0397e15
I20260807 11:28:30.513116 6743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 11:28:30 dist-test-slave-2w3w krb5kdc[6659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 11:28:30.526553 6743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca0af031e22045c88a2838a61a15c4c8: Bootstrap starting.
I20260807 11:28:30.529860 6743 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca0af031e22045c88a2838a61a15c4c8: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:30.530894 6743 log.cc:824] T 00000000000000000000000000000000 P ca0af031e22045c88a2838a61a15c4c8: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:30.533900 6743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca0af031e22045c88a2838a61a15c4c8: No bootstrap required, opened a new log
I20260807 11:28:30.534044 6746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:30.512305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55171 (local address 127.0.0.1:41041)
0807 11:28:30.513092 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:30.513106 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:30.513149 (+ 43us) server_negotiation.cc:407] Connection header received
0807 11:28:30.514799 (+ 1650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:30.514839 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:30.515290 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:30.515632 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:30.517511 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:30.518889 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:30.520745 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:30.521439 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:30.521567 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:30.526470 (+ 4903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:30.526478 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:30.526493 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:30.526539 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:30.529672 (+ 3133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:30.530540 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:30.530547 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:30.530553 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:30.530730 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:30.531229 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:30.531234 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:30.531237 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:30.531783 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:30.532132 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:30.532577 (+ 445us) server_negotiation.cc:299] Negotiation successful
0807 11:28:30.533040 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":342,"threads_started":1}
I20260807 11:28:30.537875 17172 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 11:28:30.542256 6743 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca0af031e22045c88a2838a61a15c4c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca0af031e22045c88a2838a61a15c4c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41041 } }
I20260807 11:28:30.542733 6743 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca0af031e22045c88a2838a61a15c4c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:30.542865 6743 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca0af031e22045c88a2838a61a15c4c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca0af031e22045c88a2838a61a15c4c8, State: Initialized, Role: FOLLOWER
I20260807 11:28:30.543577 6743 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca0af031e22045c88a2838a61a15c4c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca0af031e22045c88a2838a61a15c4c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41041 } }
I20260807 11:28:30.543866 6743 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca0af031e22045c88a2838a61a15c4c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:28:30.544039 6743 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca0af031e22045c88a2838a61a15c4c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:28:30.544243 6743 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca0af031e22045c88a2838a61a15c4c8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:30.547194 6743 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca0af031e22045c88a2838a61a15c4c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca0af031e22045c88a2838a61a15c4c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41041 } }
I20260807 11:28:30.547802 6743 leader_election.cc:302] T 00000000000000000000000000000000 P ca0af031e22045c88a2838a61a15c4c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca0af031e22045c88a2838a61a15c4c8; no voters:
I20260807 11:28:30.548485 6743 leader_election.cc:288] T 00000000000000000000000000000000 P ca0af031e22045c88a2838a61a15c4c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:28:30.548646 6753 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca0af031e22045c88a2838a61a15c4c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:30.550606 6743 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca0af031e22045c88a2838a61a15c4c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:28:30.552520 6753 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca0af031e22045c88a2838a61a15c4c8 [term 1 LEADER]: Becoming Leader. State: Replica: ca0af031e22045c88a2838a61a15c4c8, State: Running, Role: LEADER
I20260807 11:28:30.553282 6753 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca0af031e22045c88a2838a61a15c4c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca0af031e22045c88a2838a61a15c4c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41041 } }
I20260807 11:28:30.556489 6755 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca0af031e22045c88a2838a61a15c4c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca0af031e22045c88a2838a61a15c4c8. Latest consensus state: current_term: 1 leader_uuid: "ca0af031e22045c88a2838a61a15c4c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca0af031e22045c88a2838a61a15c4c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41041 } } }
I20260807 11:28:30.556871 6755 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca0af031e22045c88a2838a61a15c4c8 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:30.557379 6754 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca0af031e22045c88a2838a61a15c4c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca0af031e22045c88a2838a61a15c4c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca0af031e22045c88a2838a61a15c4c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41041 } } }
I20260807 11:28:30.557657 6754 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca0af031e22045c88a2838a61a15c4c8 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:30.560002 6762 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:28:30.563117 6746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:30.542315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59911 (local address 127.0.0.1:41041)
0807 11:28:30.542513 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:30.542520 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:30.543044 (+ 524us) server_negotiation.cc:407] Connection header received
0807 11:28:30.543346 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:30.543352 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:30.543431 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:30.543539 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:30.545090 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:30.546220 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:30.547580 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:30.548319 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:30.548479 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:30.552306 (+ 3827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:30.552315 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:30.552322 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:30.552375 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:30.557460 (+ 5085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:30.558553 (+ 1093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:30.558558 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:30.558561 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:30.558775 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:30.559327 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:30.559334 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:30.559340 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:30.559778 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:30.561341 (+ 1563us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:30.561512 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 11:28:30.561830 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 11:28:30.565420 6762 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:28:30.574947 6762 catalog_manager.cc:1347] Generated new cluster ID: 13207767af7d488ca7f33f61763e785b
I20260807 11:28:30.575124 6762 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:28:30.588042 6762 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:28:30.589303 6762 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:28:30.601076 6762 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca0af031e22045c88a2838a61a15c4c8: Generated new TSK 0
I20260807 11:28:30.601969 6762 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:28:30.612388 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 6673
I20260807 11:28:30.613713 6738 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:30.706512 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 6673
2026-08-07T11:28:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2107 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:28:30 dist-test-slave-2w3w krb5kdc[6776](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:28:30 dist-test-slave-2w3w krb5kdc[6776](info): set up 2 sockets
Aug 07 11:28:30 dist-test-slave-2w3w krb5kdc[6776](info): commencing operation
krb5kdc: starting...
W20260807 11:28:31.876855 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:28:32 dist-test-slave-2w3w krb5kdc[6776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:28:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:28:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 11:28:32.174310 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34405
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:32.512553 6790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:32.512948 6790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:32.513154 6790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:32.520689 6790 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:28:32.520875 6790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:32.520998 6790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:32.521107 6790 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:28:32.521194 6790 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:28:32.532212 6790 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:32.533640 6790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:32.535100 6790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:32.547526 6795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:32.547466 6798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:32.547544 6796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:32.548424 6790 server_base.cc:1034] running on GCE node
I20260807 11:28:32.549170 6790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:32.550524 6790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:32.551721 6790 hybrid_clock.cc:648] HybridClock initialized: now 1786102112551681 us; error 31 us; skew 500 ppm
I20260807 11:28:32.552139 6790 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:32 dist-test-slave-2w3w krb5kdc[6776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102112, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:32.555348 6790 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 11:28:32.556844 6790 webserver.cc:460] Webserver started at http://127.0.0.1:40263/ using document root <none> and password file <none>
I20260807 11:28:32.557488 6790 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:32.557593 6790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:32.557857 6790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:32.560871 6790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "1f714e10de7a41d385283caa9c3f797c"
format_stamp: "Formatted at 2026-08-07 11:28:32 on dist-test-slave-2w3w"
server_key: "42e5d4bfca311e44978e62c827761bd6"
server_key_iv: "3f5bbfa2936a1ca11348318656fe882a"
server_key_version: "encryptionkey@0"
I20260807 11:28:32.561611 6790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "1f714e10de7a41d385283caa9c3f797c"
format_stamp: "Formatted at 2026-08-07 11:28:32 on dist-test-slave-2w3w"
server_key: "42e5d4bfca311e44978e62c827761bd6"
server_key_iv: "3f5bbfa2936a1ca11348318656fe882a"
server_key_version: "encryptionkey@0"
I20260807 11:28:32.566816 6790 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 11:28:32.570358 6805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:32.571682 6790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:28:32.571877 6790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "1f714e10de7a41d385283caa9c3f797c"
format_stamp: "Formatted at 2026-08-07 11:28:32 on dist-test-slave-2w3w"
server_key: "42e5d4bfca311e44978e62c827761bd6"
server_key_iv: "3f5bbfa2936a1ca11348318656fe882a"
server_key_version: "encryptionkey@0"
I20260807 11:28:32.572078 6790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:32.593732 6790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:32.598145 6790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:32.598449 6790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:32.617595 6790 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34405
I20260807 11:28:32.617630 6858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34405 every 8 connection(s)
I20260807 11:28:32.619060 6790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:28:32.622706 6859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:32.623421 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 6790
I20260807 11:28:32.623612 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:28:32.624203 17172 external_mini_cluster.cc:1428] Setting key 68cffe95e01b346ebda448e20d5c31fc
I20260807 11:28:32.636813 6859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f714e10de7a41d385283caa9c3f797c: Bootstrap starting.
I20260807 11:28:32.638299 6862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:32.626095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59751 (local address 127.0.0.1:34405)
0807 11:28:32.626920 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:32.626932 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:32.627029 (+ 97us) server_negotiation.cc:407] Connection header received
0807 11:28:32.629117 (+ 2088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:32.629143 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:32.629540 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:32.629883 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:32.630532 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:32.631870 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:32.633856 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:32.634528 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:32.634655 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:32.636010 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:32.636017 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:32.636031 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:32.636574 (+ 543us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:32.636651 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:32.636675 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:32.636767 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:32.637058 (+ 291us) server_negotiation.cc:299] Negotiation successful
0807 11:28:32.637371 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":408,"threads_started":1}
I20260807 11:28:32.640123 6859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f714e10de7a41d385283caa9c3f797c: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:32.641295 6859 log.cc:824] T 00000000000000000000000000000000 P 1f714e10de7a41d385283caa9c3f797c: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:32.642153 17172 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 11:28:32.644141 6859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f714e10de7a41d385283caa9c3f797c: No bootstrap required, opened a new log
I20260807 11:28:32.653328 6862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:32.646721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52977 (local address 127.0.0.1:34405)
0807 11:28:32.646978 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:32.646985 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:32.647337 (+ 352us) server_negotiation.cc:407] Connection header received
0807 11:28:32.647489 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:32.647496 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:32.647575 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:32.647703 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:32.648206 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:32.649309 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:32.650662 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:32.651396 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:32.651542 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:32.652575 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:32.652581 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:32.652586 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:32.652609 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:32.652669 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:32.652688 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:32.652780 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:32.652904 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 11:28:32.652998 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 11:28:32.653582 6859 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f714e10de7a41d385283caa9c3f797c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f714e10de7a41d385283caa9c3f797c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34405 } }
I20260807 11:28:32.654029 6859 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f714e10de7a41d385283caa9c3f797c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:32.654155 6859 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f714e10de7a41d385283caa9c3f797c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f714e10de7a41d385283caa9c3f797c, State: Initialized, Role: FOLLOWER
I20260807 11:28:32.654757 6859 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f714e10de7a41d385283caa9c3f797c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f714e10de7a41d385283caa9c3f797c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34405 } }
I20260807 11:28:32.655053 6859 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f714e10de7a41d385283caa9c3f797c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:28:32.655207 6859 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f714e10de7a41d385283caa9c3f797c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:28:32.655388 6859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f714e10de7a41d385283caa9c3f797c [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:32.657606 6859 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f714e10de7a41d385283caa9c3f797c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f714e10de7a41d385283caa9c3f797c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34405 } }
I20260807 11:28:32.658108 6859 leader_election.cc:302] T 00000000000000000000000000000000 P 1f714e10de7a41d385283caa9c3f797c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f714e10de7a41d385283caa9c3f797c; no voters:
I20260807 11:28:32.658665 6859 leader_election.cc:288] T 00000000000000000000000000000000 P 1f714e10de7a41d385283caa9c3f797c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:28:32.658800 6869 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f714e10de7a41d385283caa9c3f797c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:32.659582 6869 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f714e10de7a41d385283caa9c3f797c [term 1 LEADER]: Becoming Leader. State: Replica: 1f714e10de7a41d385283caa9c3f797c, State: Running, Role: LEADER
I20260807 11:28:32.660140 6869 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f714e10de7a41d385283caa9c3f797c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f714e10de7a41d385283caa9c3f797c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34405 } }
I20260807 11:28:32.660885 6859 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f714e10de7a41d385283caa9c3f797c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:28:32.662949 6871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f714e10de7a41d385283caa9c3f797c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f714e10de7a41d385283caa9c3f797c. Latest consensus state: current_term: 1 leader_uuid: "1f714e10de7a41d385283caa9c3f797c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f714e10de7a41d385283caa9c3f797c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34405 } } }
I20260807 11:28:32.663216 6870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f714e10de7a41d385283caa9c3f797c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f714e10de7a41d385283caa9c3f797c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f714e10de7a41d385283caa9c3f797c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34405 } } }
I20260807 11:28:32.663563 6870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f714e10de7a41d385283caa9c3f797c [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:32.663597 6871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f714e10de7a41d385283caa9c3f797c [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:32.665783 6875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:28:32.670750 6875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:28:32.680958 6875 catalog_manager.cc:1347] Generated new cluster ID: 2aee60a3124a443f8721f6a2ca8d5ff9
I20260807 11:28:32.681134 6875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:28:32.690517 6875 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:28:32.691932 6875 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:28:32.698220 6875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f714e10de7a41d385283caa9c3f797c: Generated new TSK 0
I20260807 11:28:32.699249 6875 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:28:32.714504 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 6790
W20260807 11:28:32.715705 6854 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59751
W20260807 11:28:32.716385 17172 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 6790: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 11:28:32.716537 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 6790
2026-08-07T11:28:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2008 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:28:32 dist-test-slave-2w3w krb5kdc[6891](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:28:32 dist-test-slave-2w3w krb5kdc[6891](info): set up 2 sockets
Aug 07 11:28:32 dist-test-slave-2w3w krb5kdc[6891](info): commencing operation
krb5kdc: starting...
W20260807 11:28:33.857143 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:28:34 dist-test-slave-2w3w krb5kdc[6891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:28:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:28:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 11:28:34.162441 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34195
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:34.502034 6904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:34.502365 6904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:34.502547 6904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:34.509924 6904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:28:34.510070 6904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:34.510191 6904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:34.510295 6904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:28:34.510368 6904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:28:34.520442 6904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:34.521773 6904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:34.523142 6904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:34.535600 6910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:34.535652 6913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:34.535691 6911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:34.536514 6904 server_base.cc:1034] running on GCE node
I20260807 11:28:34.537388 6904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:34.538897 6904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:34.540172 6904 hybrid_clock.cc:648] HybridClock initialized: now 1786102114540134 us; error 38 us; skew 500 ppm
I20260807 11:28:34.540551 6904 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:34 dist-test-slave-2w3w krb5kdc[6891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102114, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:34.547248 6904 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 11:28:34.549048 6904 webserver.cc:460] Webserver started at http://127.0.0.1:43259/ using document root <none> and password file <none>
I20260807 11:28:34.549718 6904 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:34.549821 6904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:34.550076 6904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:34.553119 6904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "d113aa5ddf504787b4d3f6d354ee9499"
format_stamp: "Formatted at 2026-08-07 11:28:34 on dist-test-slave-2w3w"
server_key: "d866d78229e98c3d85b67e6f8eed5562"
server_key_iv: "9dbce0f50005de0fa6b1df25ed354343"
server_key_version: "encryptionkey@0"
I20260807 11:28:34.553946 6904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "d113aa5ddf504787b4d3f6d354ee9499"
format_stamp: "Formatted at 2026-08-07 11:28:34 on dist-test-slave-2w3w"
server_key: "d866d78229e98c3d85b67e6f8eed5562"
server_key_iv: "9dbce0f50005de0fa6b1df25ed354343"
server_key_version: "encryptionkey@0"
I20260807 11:28:34.559221 6904 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260807 11:28:34.562539 6921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:34.563964 6904 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:28:34.564143 6904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "d113aa5ddf504787b4d3f6d354ee9499"
format_stamp: "Formatted at 2026-08-07 11:28:34 on dist-test-slave-2w3w"
server_key: "d866d78229e98c3d85b67e6f8eed5562"
server_key_iv: "9dbce0f50005de0fa6b1df25ed354343"
server_key_version: "encryptionkey@0"
I20260807 11:28:34.564324 6904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:34.570345 6904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:34.574484 6904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:34.574807 6904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:34.594506 6904 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34195
I20260807 11:28:34.594549 6976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34195 every 8 connection(s)
I20260807 11:28:34.596081 6904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:28:34.599843 6977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:34.601811 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 6904
I20260807 11:28:34.602037 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:28:34.602581 17172 external_mini_cluster.cc:1428] Setting key f24cfda803c3a617af9c5445a4c77f48
I20260807 11:28:34.611322 6980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:34.604493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34923 (local address 127.0.0.1:34195)
0807 11:28:34.605337 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:34.605354 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:34.605417 (+ 63us) server_negotiation.cc:407] Connection header received
0807 11:28:34.607261 (+ 1844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:34.607293 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:34.607790 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:34.608314 (+ 524us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:34.609289 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:34.609297 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:34.609315 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:34.609331 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:34.609403 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:34.609430 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:34.609648 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:34.609777 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 11:28:34.609989 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":346,"threads_started":1}
I20260807 11:28:34.615363 17172 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 11:28:34.615324 6977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d113aa5ddf504787b4d3f6d354ee9499: Bootstrap starting.
I20260807 11:28:34.619577 6977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d113aa5ddf504787b4d3f6d354ee9499: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:34.621037 6977 log.cc:824] T 00000000000000000000000000000000 P d113aa5ddf504787b4d3f6d354ee9499: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:34.623642 6980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:34.620786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46127 (local address 127.0.0.1:34195)
0807 11:28:34.620999 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:34.621005 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:34.621509 (+ 504us) server_negotiation.cc:407] Connection header received
0807 11:28:34.621721 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:34.621729 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:34.621804 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:34.621917 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:34.623004 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:34.623009 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:34.623014 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:34.623034 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:34.623080 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:34.623102 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:34.623226 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:34.623297 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 11:28:34.623358 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 11:28:34.624841 6977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d113aa5ddf504787b4d3f6d354ee9499: No bootstrap required, opened a new log
I20260807 11:28:34.633410 6977 raft_consensus.cc:348] T 00000000000000000000000000000000 P d113aa5ddf504787b4d3f6d354ee9499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d113aa5ddf504787b4d3f6d354ee9499" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34195 } }
I20260807 11:28:34.633798 6977 raft_consensus.cc:374] T 00000000000000000000000000000000 P d113aa5ddf504787b4d3f6d354ee9499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:34.633908 6977 raft_consensus.cc:729] T 00000000000000000000000000000000 P d113aa5ddf504787b4d3f6d354ee9499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d113aa5ddf504787b4d3f6d354ee9499, State: Initialized, Role: FOLLOWER
I20260807 11:28:34.634579 6977 consensus_queue.cc:260] T 00000000000000000000000000000000 P d113aa5ddf504787b4d3f6d354ee9499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d113aa5ddf504787b4d3f6d354ee9499" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34195 } }
I20260807 11:28:34.634846 6977 raft_consensus.cc:388] T 00000000000000000000000000000000 P d113aa5ddf504787b4d3f6d354ee9499 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:28:34.635046 6977 raft_consensus.cc:482] T 00000000000000000000000000000000 P d113aa5ddf504787b4d3f6d354ee9499 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:28:34.635234 6977 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d113aa5ddf504787b4d3f6d354ee9499 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:34.637931 6977 raft_consensus.cc:504] T 00000000000000000000000000000000 P d113aa5ddf504787b4d3f6d354ee9499 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d113aa5ddf504787b4d3f6d354ee9499" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34195 } }
I20260807 11:28:34.638522 6977 leader_election.cc:302] T 00000000000000000000000000000000 P d113aa5ddf504787b4d3f6d354ee9499 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d113aa5ddf504787b4d3f6d354ee9499; no voters:
I20260807 11:28:34.639170 6977 leader_election.cc:288] T 00000000000000000000000000000000 P d113aa5ddf504787b4d3f6d354ee9499 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:28:34.639324 6987 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d113aa5ddf504787b4d3f6d354ee9499 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:34.640115 6987 raft_consensus.cc:686] T 00000000000000000000000000000000 P d113aa5ddf504787b4d3f6d354ee9499 [term 1 LEADER]: Becoming Leader. State: Replica: d113aa5ddf504787b4d3f6d354ee9499, State: Running, Role: LEADER
I20260807 11:28:34.640669 6987 consensus_queue.cc:237] T 00000000000000000000000000000000 P d113aa5ddf504787b4d3f6d354ee9499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d113aa5ddf504787b4d3f6d354ee9499" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34195 } }
I20260807 11:28:34.641396 6977 sys_catalog.cc:564] T 00000000000000000000000000000000 P d113aa5ddf504787b4d3f6d354ee9499 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:28:34.643757 6989 sys_catalog.cc:455] T 00000000000000000000000000000000 P d113aa5ddf504787b4d3f6d354ee9499 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d113aa5ddf504787b4d3f6d354ee9499. Latest consensus state: current_term: 1 leader_uuid: "d113aa5ddf504787b4d3f6d354ee9499" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d113aa5ddf504787b4d3f6d354ee9499" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34195 } } }
I20260807 11:28:34.643705 6988 sys_catalog.cc:455] T 00000000000000000000000000000000 P d113aa5ddf504787b4d3f6d354ee9499 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d113aa5ddf504787b4d3f6d354ee9499" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d113aa5ddf504787b4d3f6d354ee9499" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34195 } } }
I20260807 11:28:34.644306 6988 sys_catalog.cc:458] T 00000000000000000000000000000000 P d113aa5ddf504787b4d3f6d354ee9499 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:34.646193 6989 sys_catalog.cc:458] T 00000000000000000000000000000000 P d113aa5ddf504787b4d3f6d354ee9499 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:34.646739 6995 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:28:34.652392 6995 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:28:34.661520 6995 catalog_manager.cc:1347] Generated new cluster ID: bb8df5b150bc48259f5d16a80310408b
I20260807 11:28:34.661656 6995 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:28:34.689684 6995 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:28:34.690141 6995 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:28:34.702132 6995 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d113aa5ddf504787b4d3f6d354ee9499: Generated new TSK 0
I20260807 11:28:34.703074 6995 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:28:34.713557 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 6904
W20260807 11:28:34.714774 6972 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34923
W20260807 11:28:34.715550 17172 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 6904: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 11:28:34.715723 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 6904
2026-08-07T11:28:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1999 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:28:34 dist-test-slave-2w3w krb5kdc[7009](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:28:34 dist-test-slave-2w3w krb5kdc[7009](info): set up 2 sockets
Aug 07 11:28:34 dist-test-slave-2w3w krb5kdc[7009](info): commencing operation
krb5kdc: starting...
W20260807 11:28:35.873075 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:28:36 dist-test-slave-2w3w krb5kdc[7009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:28:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:28:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 11:28:36.174316 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42773
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:36.517756 7022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:36.518045 7022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:36.518221 7022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:36.525485 7022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:28:36.525627 7022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:36.525784 7022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:36.525889 7022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:28:36.525981 7022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:28:36.535977 7022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:36.537329 7022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:36.538647 7022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:36.550879 7030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:36.550865 7028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:36.551497 7027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:36.551764 7022 server_base.cc:1034] running on GCE node
I20260807 11:28:36.552609 7022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:36.554109 7022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:36.555323 7022 hybrid_clock.cc:648] HybridClock initialized: now 1786102116555287 us; error 37 us; skew 500 ppm
I20260807 11:28:36.555696 7022 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:36 dist-test-slave-2w3w krb5kdc[7009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102116, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:36.558924 7022 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 11:28:36.560508 7022 webserver.cc:460] Webserver started at http://127.0.0.1:38099/ using document root <none> and password file <none>
I20260807 11:28:36.561188 7022 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:36.561339 7022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:36.561654 7022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:36.564713 7022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "94c8fa42430a4a5384e466b92c1be734"
format_stamp: "Formatted at 2026-08-07 11:28:36 on dist-test-slave-2w3w"
server_key: "74f46ec854afb33d4feb9117b224c0b1"
server_key_iv: "27271794287d52f455824c238a481773"
server_key_version: "encryptionkey@0"
I20260807 11:28:36.565495 7022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "94c8fa42430a4a5384e466b92c1be734"
format_stamp: "Formatted at 2026-08-07 11:28:36 on dist-test-slave-2w3w"
server_key: "74f46ec854afb33d4feb9117b224c0b1"
server_key_iv: "27271794287d52f455824c238a481773"
server_key_version: "encryptionkey@0"
I20260807 11:28:36.570643 7022 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 11:28:36.574084 7037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:36.575490 7022 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260807 11:28:36.575670 7022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "94c8fa42430a4a5384e466b92c1be734"
format_stamp: "Formatted at 2026-08-07 11:28:36 on dist-test-slave-2w3w"
server_key: "74f46ec854afb33d4feb9117b224c0b1"
server_key_iv: "27271794287d52f455824c238a481773"
server_key_version: "encryptionkey@0"
I20260807 11:28:36.575867 7022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:36.610306 7022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:36.614676 7022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:36.615163 7022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:36.634347 7022 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42773
I20260807 11:28:36.634380 7089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42773 every 8 connection(s)
I20260807 11:28:36.635934 7022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:28:36.640058 7090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:36.643479 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 7022
I20260807 11:28:36.643720 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:28:36.644170 17172 external_mini_cluster.cc:1428] Setting key 5ede44e27e85991765c1bb3d980eea9b
I20260807 11:28:36.652262 7090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94c8fa42430a4a5384e466b92c1be734: Bootstrap starting.
I20260807 11:28:36.655512 7090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94c8fa42430a4a5384e466b92c1be734: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:36.656597 7090 log.cc:824] T 00000000000000000000000000000000 P 94c8fa42430a4a5384e466b92c1be734: Log is configured to *not* fsync() on all Append() calls
Aug 07 11:28:36 dist-test-slave-2w3w krb5kdc[7009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 11:28:36.659786 7090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94c8fa42430a4a5384e466b92c1be734: No bootstrap required, opened a new log
I20260807 11:28:36.668015 7094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:36.646165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:33417 (local address 127.0.0.1:42773)
0807 11:28:36.647020 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:36.647039 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:36.647096 (+ 57us) server_negotiation.cc:407] Connection header received
0807 11:28:36.648807 (+ 1711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:36.648828 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:36.649239 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:36.649654 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:36.651269 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:36.652622 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:36.653913 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:36.654563 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:36.654801 (+ 238us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:36.659297 (+ 4496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:36.659309 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:36.659328 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:36.659371 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:36.663408 (+ 4037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:36.664213 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:36.664221 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:36.664228 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:36.664447 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:36.664856 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:36.664861 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:36.664864 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:36.665562 (+ 698us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:36.665900 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:36.666233 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 11:28:36.666607 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":360,"threads_started":1}
I20260807 11:28:36.670620 7090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94c8fa42430a4a5384e466b92c1be734 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c8fa42430a4a5384e466b92c1be734" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42773 } }
I20260807 11:28:36.671077 7090 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94c8fa42430a4a5384e466b92c1be734 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:36.671177 7090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94c8fa42430a4a5384e466b92c1be734 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94c8fa42430a4a5384e466b92c1be734, State: Initialized, Role: FOLLOWER
I20260807 11:28:36.671789 7090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94c8fa42430a4a5384e466b92c1be734 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c8fa42430a4a5384e466b92c1be734" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42773 } }
I20260807 11:28:36.672076 7090 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94c8fa42430a4a5384e466b92c1be734 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:28:36.672158 17172 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 11:28:36.672221 7090 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94c8fa42430a4a5384e466b92c1be734 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:28:36.672392 7090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94c8fa42430a4a5384e466b92c1be734 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:36.675596 7090 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94c8fa42430a4a5384e466b92c1be734 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c8fa42430a4a5384e466b92c1be734" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42773 } }
I20260807 11:28:36.676270 7090 leader_election.cc:302] T 00000000000000000000000000000000 P 94c8fa42430a4a5384e466b92c1be734 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94c8fa42430a4a5384e466b92c1be734; no voters:
I20260807 11:28:36.676908 7090 leader_election.cc:288] T 00000000000000000000000000000000 P 94c8fa42430a4a5384e466b92c1be734 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:28:36.678215 7100 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94c8fa42430a4a5384e466b92c1be734 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:36.679168 7100 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94c8fa42430a4a5384e466b92c1be734 [term 1 LEADER]: Becoming Leader. State: Replica: 94c8fa42430a4a5384e466b92c1be734, State: Running, Role: LEADER
I20260807 11:28:36.679174 7090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94c8fa42430a4a5384e466b92c1be734 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:28:36.679903 7100 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94c8fa42430a4a5384e466b92c1be734 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c8fa42430a4a5384e466b92c1be734" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42773 } }
I20260807 11:28:36.683508 7103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94c8fa42430a4a5384e466b92c1be734 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94c8fa42430a4a5384e466b92c1be734. Latest consensus state: current_term: 1 leader_uuid: "94c8fa42430a4a5384e466b92c1be734" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c8fa42430a4a5384e466b92c1be734" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42773 } } }
I20260807 11:28:36.683907 7103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94c8fa42430a4a5384e466b92c1be734 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:36.684473 7101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94c8fa42430a4a5384e466b92c1be734 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94c8fa42430a4a5384e466b92c1be734" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94c8fa42430a4a5384e466b92c1be734" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42773 } } }
I20260807 11:28:36.684821 7101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94c8fa42430a4a5384e466b92c1be734 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:36.685997 7107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:28:36.699580 7094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:36.677625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:38997 (local address 127.0.0.1:42773)
0807 11:28:36.677793 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:36.677799 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:36.678631 (+ 832us) server_negotiation.cc:407] Connection header received
0807 11:28:36.678706 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:36.678717 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:36.678826 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:36.678939 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:36.681140 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:36.682933 (+ 1793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:36.685584 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:36.686553 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:36.687640 (+ 1087us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:36.691108 (+ 3468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:36.691121 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:36.691130 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:36.691187 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:36.696136 (+ 4949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:36.697157 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:36.697162 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:36.697165 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:36.697329 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:36.697869 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:36.697874 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:36.697878 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:36.698313 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:36.698614 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:36.699008 (+ 394us) server_negotiation.cc:299] Negotiation successful
0807 11:28:36.699210 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 11:28:36.705638 7107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:28:36.715755 7107 catalog_manager.cc:1347] Generated new cluster ID: 5e2897f7561244ad806385c881b5da09
I20260807 11:28:36.715906 7107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:28:36.734663 7107 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:28:36.736747 7107 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:28:36.743309 7107 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94c8fa42430a4a5384e466b92c1be734: Generated new TSK 0
I20260807 11:28:36.744438 7107 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:28:36.749711 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 7022
I20260807 11:28:36.751462 7085 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:36.844749 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 7022
W20260807 11:28:36.864347 7007 connection.cc:535] client connection to 127.0.0.1:42773 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42773): Connection reset by peer (error 104)
2026-08-07T11:28:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2129 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:28:36 dist-test-slave-2w3w krb5kdc[7125](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:28:36 dist-test-slave-2w3w krb5kdc[7125](info): set up 2 sockets
Aug 07 11:28:36 dist-test-slave-2w3w krb5kdc[7125](info): commencing operation
krb5kdc: starting...
W20260807 11:28:37.997566 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:28:38 dist-test-slave-2w3w krb5kdc[7125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:28:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:28:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 11:28:38.297986 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39431
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:38.638567 7139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:38.638892 7139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:38.639124 7139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:38.646479 7139 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:28:38.646620 7139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:38.646715 7139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:38.646831 7139 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:28:38.646920 7139 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:28:38.657100 7139 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:38.658511 7139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:38.659866 7139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:38.672092 7145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:38.672181 7148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:38.672133 7146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:38.673127 7139 server_base.cc:1034] running on GCE node
I20260807 11:28:38.674088 7139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:38.675716 7139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:38.676970 7139 hybrid_clock.cc:648] HybridClock initialized: now 1786102118676923 us; error 60 us; skew 500 ppm
I20260807 11:28:38.677346 7139 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:38 dist-test-slave-2w3w krb5kdc[7125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102118, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:38.680752 7139 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 11:28:38.682391 7139 webserver.cc:460] Webserver started at http://127.0.0.1:46813/ using document root <none> and password file <none>
I20260807 11:28:38.683228 7139 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:38.683382 7139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:38.683704 7139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:38.686932 7139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "00587f3573c543dfa2f084794d130d38"
format_stamp: "Formatted at 2026-08-07 11:28:38 on dist-test-slave-2w3w"
server_key: "df35fbbd8d9c436995581dd5450f3b0a"
server_key_iv: "33e61427cf02783cca49ea3ed6c769e6"
server_key_version: "encryptionkey@0"
I20260807 11:28:38.687848 7139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "00587f3573c543dfa2f084794d130d38"
format_stamp: "Formatted at 2026-08-07 11:28:38 on dist-test-slave-2w3w"
server_key: "df35fbbd8d9c436995581dd5450f3b0a"
server_key_iv: "33e61427cf02783cca49ea3ed6c769e6"
server_key_version: "encryptionkey@0"
I20260807 11:28:38.693413 7139 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 11:28:38.697045 7155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:38.698388 7139 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 11:28:38.698585 7139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "00587f3573c543dfa2f084794d130d38"
format_stamp: "Formatted at 2026-08-07 11:28:38 on dist-test-slave-2w3w"
server_key: "df35fbbd8d9c436995581dd5450f3b0a"
server_key_iv: "33e61427cf02783cca49ea3ed6c769e6"
server_key_version: "encryptionkey@0"
I20260807 11:28:38.698793 7139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:38.721494 7139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:38.726331 7139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:38.726648 7139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:38.745829 7139 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39431
I20260807 11:28:38.745862 7206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39431 every 8 connection(s)
I20260807 11:28:38.747282 7139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:28:38.749307 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 7139
I20260807 11:28:38.749523 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:28:38.750130 17172 external_mini_cluster.cc:1428] Setting key f51fd197a7b66943bf7237ff6f251120
I20260807 11:28:38.751791 7207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:38.763870 7210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:38.752054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:58937 (local address 127.0.0.1:39431)
0807 11:28:38.752869 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:38.752880 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:38.752921 (+ 41us) server_negotiation.cc:407] Connection header received
0807 11:28:38.754557 (+ 1636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:38.754579 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:38.755055 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:38.755428 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:38.756031 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:38.757423 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:38.759318 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:38.760011 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:38.760142 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:38.761513 (+ 1371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:38.761520 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:38.761533 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:38.762091 (+ 558us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:38.762164 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:38.762190 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:38.762310 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:38.762630 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 11:28:38.762812 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":375,"threads_started":1}
I20260807 11:28:38.765465 7207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00587f3573c543dfa2f084794d130d38: Bootstrap starting.
I20260807 11:28:38.768033 17172 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 11:28:38.768978 7207 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00587f3573c543dfa2f084794d130d38: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:38.769999 7207 log.cc:824] T 00000000000000000000000000000000 P 00587f3573c543dfa2f084794d130d38: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:38.773108 7207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00587f3573c543dfa2f084794d130d38: No bootstrap required, opened a new log
I20260807 11:28:38.779618 7210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:38.772656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:51165 (local address 127.0.0.1:39431)
0807 11:28:38.772833 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:38.772838 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:38.773374 (+ 536us) server_negotiation.cc:407] Connection header received
0807 11:28:38.773558 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:38.773565 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:38.773660 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:38.773771 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:38.774285 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:38.775540 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:38.776856 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:38.777564 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:38.777685 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:38.778797 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:38.778804 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:38.778808 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:38.778830 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:38.778879 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:38.778898 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:38.779040 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:38.779226 (+ 186us) server_negotiation.cc:299] Negotiation successful
0807 11:28:38.779286 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 11:28:38.782550 7207 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00587f3573c543dfa2f084794d130d38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00587f3573c543dfa2f084794d130d38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39431 } }
I20260807 11:28:38.783043 7207 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00587f3573c543dfa2f084794d130d38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:38.783179 7207 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00587f3573c543dfa2f084794d130d38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00587f3573c543dfa2f084794d130d38, State: Initialized, Role: FOLLOWER
I20260807 11:28:38.783756 7207 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00587f3573c543dfa2f084794d130d38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00587f3573c543dfa2f084794d130d38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39431 } }
I20260807 11:28:38.784000 7207 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00587f3573c543dfa2f084794d130d38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:28:38.784145 7207 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00587f3573c543dfa2f084794d130d38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:28:38.784328 7207 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00587f3573c543dfa2f084794d130d38 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:38.786512 7207 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00587f3573c543dfa2f084794d130d38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00587f3573c543dfa2f084794d130d38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39431 } }
I20260807 11:28:38.787082 7207 leader_election.cc:302] T 00000000000000000000000000000000 P 00587f3573c543dfa2f084794d130d38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00587f3573c543dfa2f084794d130d38; no voters:
I20260807 11:28:38.787741 7207 leader_election.cc:288] T 00000000000000000000000000000000 P 00587f3573c543dfa2f084794d130d38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:28:38.787889 7219 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00587f3573c543dfa2f084794d130d38 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:38.788625 7219 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00587f3573c543dfa2f084794d130d38 [term 1 LEADER]: Becoming Leader. State: Replica: 00587f3573c543dfa2f084794d130d38, State: Running, Role: LEADER
I20260807 11:28:38.789260 7219 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00587f3573c543dfa2f084794d130d38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00587f3573c543dfa2f084794d130d38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39431 } }
I20260807 11:28:38.790205 7207 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00587f3573c543dfa2f084794d130d38 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:28:38.792367 7220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00587f3573c543dfa2f084794d130d38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00587f3573c543dfa2f084794d130d38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00587f3573c543dfa2f084794d130d38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39431 } } }
I20260807 11:28:38.792824 7220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00587f3573c543dfa2f084794d130d38 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:38.793577 7221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00587f3573c543dfa2f084794d130d38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00587f3573c543dfa2f084794d130d38. Latest consensus state: current_term: 1 leader_uuid: "00587f3573c543dfa2f084794d130d38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00587f3573c543dfa2f084794d130d38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39431 } } }
I20260807 11:28:38.793860 7221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00587f3573c543dfa2f084794d130d38 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:38.796247 7228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:28:38.801514 7228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:28:38.811136 7228 catalog_manager.cc:1347] Generated new cluster ID: 99d26d8679b644debd86fdaab40c47e2
I20260807 11:28:38.811272 7228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:28:38.827309 7228 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:28:38.828646 7228 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:28:38.839958 7228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00587f3573c543dfa2f084794d130d38: Generated new TSK 0
I20260807 11:28:38.840790 7228 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:28:38.853411 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 7139
W20260807 11:28:38.854571 7202 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.84:58937
W20260807 11:28:38.855404 17172 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 7139: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 11:28:38.855568 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 7139
W20260807 11:28:38.875274 7123 connection.cc:535] client connection to 127.0.0.1:39431 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39431): Connection reset by peer (error 104)
2026-08-07T11:28:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2010 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:28:38 dist-test-slave-2w3w krb5kdc[7241](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:28:38 dist-test-slave-2w3w krb5kdc[7241](info): set up 2 sockets
Aug 07 11:28:38 dist-test-slave-2w3w krb5kdc[7241](info): commencing operation
krb5kdc: starting...
W20260807 11:28:40.001245 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.001s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:28:40 dist-test-slave-2w3w krb5kdc[7241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:28:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:28:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 11:28:40.307981 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35437
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:40.650720 7254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:40.651120 7254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:40.651301 7254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:40.658540 7254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:28:40.658700 7254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:40.658826 7254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:40.658936 7254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:28:40.659041 7254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:28:40.669829 7254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:40.671257 7254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:40.672608 7254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:40.684669 7261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:40.684762 7260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:40.684774 7263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:40.685334 7254 server_base.cc:1034] running on GCE node
I20260807 11:28:40.686131 7254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:40.687656 7254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:40.688884 7254 hybrid_clock.cc:648] HybridClock initialized: now 1786102120688849 us; error 24 us; skew 500 ppm
I20260807 11:28:40.689247 7254 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:40 dist-test-slave-2w3w krb5kdc[7241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102120, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:40.701983 7254 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 11:28:40.703696 7254 webserver.cc:460] Webserver started at http://127.0.0.1:46503/ using document root <none> and password file <none>
I20260807 11:28:40.704432 7254 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:40.704583 7254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:40.704892 7254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:40.708139 7254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "bb20623a074246e08cfba5d6372b4d4f"
format_stamp: "Formatted at 2026-08-07 11:28:40 on dist-test-slave-2w3w"
server_key: "4404e6dca65f8abe8c7d2c8b938c191d"
server_key_iv: "18e508ede33c95987cdd0a710bcb8751"
server_key_version: "encryptionkey@0"
I20260807 11:28:40.708979 7254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "bb20623a074246e08cfba5d6372b4d4f"
format_stamp: "Formatted at 2026-08-07 11:28:40 on dist-test-slave-2w3w"
server_key: "4404e6dca65f8abe8c7d2c8b938c191d"
server_key_iv: "18e508ede33c95987cdd0a710bcb8751"
server_key_version: "encryptionkey@0"
I20260807 11:28:40.714315 7254 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 11:28:40.718045 7270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:40.719476 7254 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:28:40.719671 7254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "bb20623a074246e08cfba5d6372b4d4f"
format_stamp: "Formatted at 2026-08-07 11:28:40 on dist-test-slave-2w3w"
server_key: "4404e6dca65f8abe8c7d2c8b938c191d"
server_key_iv: "18e508ede33c95987cdd0a710bcb8751"
server_key_version: "encryptionkey@0"
I20260807 11:28:40.719888 7254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:40.725806 7254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:40.729940 7254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:40.730237 7254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:40.749439 7254 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35437
I20260807 11:28:40.749496 7325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35437 every 8 connection(s)
I20260807 11:28:40.750859 7254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:28:40.755184 7326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:40.759038 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 7254
I20260807 11:28:40.759227 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:28:40.759694 17172 external_mini_cluster.cc:1428] Setting key 6e2eccf68c75a094a65706a1b9a63337
I20260807 11:28:40.768514 7329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:40.761587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:37817 (local address 127.0.0.1:35437)
0807 11:28:40.762415 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:40.762426 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:40.762468 (+ 42us) server_negotiation.cc:407] Connection header received
0807 11:28:40.764148 (+ 1680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:40.764171 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:40.764533 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:40.764877 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:40.766018 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:40.766024 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:40.766048 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:40.766065 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:40.766121 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:40.766144 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:40.766984 (+ 840us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:40.767210 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 11:28:40.767551 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":364,"threads_started":1}
I20260807 11:28:40.768558 7326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb20623a074246e08cfba5d6372b4d4f: Bootstrap starting.
I20260807 11:28:40.771751 7326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb20623a074246e08cfba5d6372b4d4f: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:40.772204 17172 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 11:28:40.772853 7326 log.cc:824] T 00000000000000000000000000000000 P bb20623a074246e08cfba5d6372b4d4f: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:40.775697 7326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb20623a074246e08cfba5d6372b4d4f: No bootstrap required, opened a new log
I20260807 11:28:40.779707 7329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:40.777196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:55685 (local address 127.0.0.1:35437)
0807 11:28:40.777396 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:40.777401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:40.777711 (+ 310us) server_negotiation.cc:407] Connection header received
0807 11:28:40.777891 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:40.777895 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:40.777947 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:40.778025 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:40.779040 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:40.779047 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:40.779050 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:40.779066 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:40.779104 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:40.779119 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:40.779211 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:40.779351 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 11:28:40.779397 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 11:28:40.784293 7326 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb20623a074246e08cfba5d6372b4d4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb20623a074246e08cfba5d6372b4d4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35437 } }
I20260807 11:28:40.784674 7326 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb20623a074246e08cfba5d6372b4d4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:40.784785 7326 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb20623a074246e08cfba5d6372b4d4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb20623a074246e08cfba5d6372b4d4f, State: Initialized, Role: FOLLOWER
I20260807 11:28:40.785383 7326 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb20623a074246e08cfba5d6372b4d4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb20623a074246e08cfba5d6372b4d4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35437 } }
I20260807 11:28:40.785642 7326 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb20623a074246e08cfba5d6372b4d4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:28:40.785785 7326 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb20623a074246e08cfba5d6372b4d4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:28:40.785970 7326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb20623a074246e08cfba5d6372b4d4f [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:40.788574 7326 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb20623a074246e08cfba5d6372b4d4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb20623a074246e08cfba5d6372b4d4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35437 } }
I20260807 11:28:40.789094 7326 leader_election.cc:302] T 00000000000000000000000000000000 P bb20623a074246e08cfba5d6372b4d4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb20623a074246e08cfba5d6372b4d4f; no voters:
I20260807 11:28:40.789727 7326 leader_election.cc:288] T 00000000000000000000000000000000 P bb20623a074246e08cfba5d6372b4d4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:28:40.789862 7337 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb20623a074246e08cfba5d6372b4d4f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:40.790598 7337 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb20623a074246e08cfba5d6372b4d4f [term 1 LEADER]: Becoming Leader. State: Replica: bb20623a074246e08cfba5d6372b4d4f, State: Running, Role: LEADER
I20260807 11:28:40.791186 7337 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb20623a074246e08cfba5d6372b4d4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb20623a074246e08cfba5d6372b4d4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35437 } }
I20260807 11:28:40.792124 7326 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb20623a074246e08cfba5d6372b4d4f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:28:40.794150 7339 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb20623a074246e08cfba5d6372b4d4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb20623a074246e08cfba5d6372b4d4f. Latest consensus state: current_term: 1 leader_uuid: "bb20623a074246e08cfba5d6372b4d4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb20623a074246e08cfba5d6372b4d4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35437 } } }
I20260807 11:28:40.794117 7338 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb20623a074246e08cfba5d6372b4d4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb20623a074246e08cfba5d6372b4d4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb20623a074246e08cfba5d6372b4d4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35437 } } }
I20260807 11:28:40.794603 7339 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb20623a074246e08cfba5d6372b4d4f [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:40.794649 7338 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb20623a074246e08cfba5d6372b4d4f [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:40.796347 7343 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:28:40.801553 7343 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:28:40.812901 7343 catalog_manager.cc:1347] Generated new cluster ID: 11216439ba234174a39008722632d5d4
I20260807 11:28:40.813025 7343 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:28:40.872848 7343 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:28:40.873385 7343 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:28:40.888865 7343 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb20623a074246e08cfba5d6372b4d4f: Generated new TSK 0
I20260807 11:28:40.889763 7343 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:28:40.906235 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 7254
W20260807 11:28:40.907400 7321 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.84:37817
W20260807 11:28:40.908217 17172 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 7254: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 11:28:40.908376 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 7254
2026-08-07T11:28:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2053 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:28:41 dist-test-slave-2w3w krb5kdc[7359](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:28:41 dist-test-slave-2w3w krb5kdc[7359](info): set up 2 sockets
Aug 07 11:28:41 dist-test-slave-2w3w krb5kdc[7359](info): commencing operation
krb5kdc: starting...
W20260807 11:28:42.076910 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:28:42 dist-test-slave-2w3w krb5kdc[7359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:28:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:28:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 11:28:42.374511 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40217
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40217
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:42.712265 7373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:42.712563 7373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:42.712754 7373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:42.720225 7373 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:28:42.720363 7373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:42.720474 7373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:42.720579 7373 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:28:42.720644 7373 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:28:42.731431 7373 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40217
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:42.732800 7373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:42.734141 7373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:42.745848 7378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:42.745858 7381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:42.745862 7379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:42.747392 7373 server_base.cc:1034] running on GCE node
I20260807 11:28:42.748221 7373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:42.749619 7373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:42.750840 7373 hybrid_clock.cc:648] HybridClock initialized: now 1786102122750795 us; error 43 us; skew 500 ppm
I20260807 11:28:42.751221 7373 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:42 dist-test-slave-2w3w krb5kdc[7359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102122, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:42.754189 7373 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 11:28:42.755729 7373 webserver.cc:460] Webserver started at http://127.0.0.1:32771/ using document root <none> and password file <none>
I20260807 11:28:42.756386 7373 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:42.756544 7373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:42.756852 7373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:42.759958 7373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "fa674b3b50394f538d7ff773126ccf13"
format_stamp: "Formatted at 2026-08-07 11:28:42 on dist-test-slave-2w3w"
server_key: "18767ce2760fadc5f8a13b68a894e205"
server_key_iv: "a85f0de9bf46698356a35c0105ab7fe0"
server_key_version: "encryptionkey@0"
I20260807 11:28:42.760731 7373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "fa674b3b50394f538d7ff773126ccf13"
format_stamp: "Formatted at 2026-08-07 11:28:42 on dist-test-slave-2w3w"
server_key: "18767ce2760fadc5f8a13b68a894e205"
server_key_iv: "a85f0de9bf46698356a35c0105ab7fe0"
server_key_version: "encryptionkey@0"
I20260807 11:28:42.765887 7373 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 11:28:42.769387 7388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:42.770689 7373 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260807 11:28:42.770887 7373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "fa674b3b50394f538d7ff773126ccf13"
format_stamp: "Formatted at 2026-08-07 11:28:42 on dist-test-slave-2w3w"
server_key: "18767ce2760fadc5f8a13b68a894e205"
server_key_iv: "a85f0de9bf46698356a35c0105ab7fe0"
server_key_version: "encryptionkey@0"
I20260807 11:28:42.771162 7373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:42.791555 7373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:42.795543 7373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:42.795840 7373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:42.814818 7373 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40217
I20260807 11:28:42.814839 7442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40217 every 8 connection(s)
I20260807 11:28:42.816248 7373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:28:42.819953 7443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:42.822198 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 7373
I20260807 11:28:42.822403 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:28:42.823004 17172 external_mini_cluster.cc:1428] Setting key 325c56c85c2587efd28b114282bec82f
I20260807 11:28:42.832829 7443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa674b3b50394f538d7ff773126ccf13: Bootstrap starting.
I20260807 11:28:42.836953 7443 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa674b3b50394f538d7ff773126ccf13: Neither blocks nor log segments found. Creating new log.
Aug 07 11:28:42 dist-test-slave-2w3w krb5kdc[7359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 11:28:42.838186 7443 log.cc:824] T 00000000000000000000000000000000 P fa674b3b50394f538d7ff773126ccf13: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:42.841784 7443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa674b3b50394f538d7ff773126ccf13: No bootstrap required, opened a new log
I20260807 11:28:42.848126 7447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:42.824857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:33133 (local address 127.0.0.1:40217)
0807 11:28:42.825702 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:42.825717 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:42.825758 (+ 41us) server_negotiation.cc:407] Connection header received
0807 11:28:42.827538 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:42.827560 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:42.827958 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:42.828279 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:42.830078 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:42.831495 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:42.833281 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:42.833947 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:42.834078 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:42.839251 (+ 5173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:42.839264 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:42.839279 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:42.839316 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:42.843280 (+ 3964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:42.844084 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:42.844098 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:42.844105 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:42.844308 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:42.844733 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:42.844738 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:42.844741 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:42.845438 (+ 697us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:42.845758 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:42.846175 (+ 417us) server_negotiation.cc:299] Negotiation successful
0807 11:28:42.846528 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":412,"threads_started":1}
I20260807 11:28:42.852073 17172 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 11:28:42.852394 7443 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa674b3b50394f538d7ff773126ccf13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa674b3b50394f538d7ff773126ccf13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40217 } }
I20260807 11:28:42.852766 7443 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa674b3b50394f538d7ff773126ccf13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:42.852880 7443 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa674b3b50394f538d7ff773126ccf13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa674b3b50394f538d7ff773126ccf13, State: Initialized, Role: FOLLOWER
I20260807 11:28:42.853451 7443 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa674b3b50394f538d7ff773126ccf13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa674b3b50394f538d7ff773126ccf13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40217 } }
I20260807 11:28:42.853708 7443 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa674b3b50394f538d7ff773126ccf13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:28:42.853868 7443 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa674b3b50394f538d7ff773126ccf13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:28:42.854071 7443 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa674b3b50394f538d7ff773126ccf13 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:42.856406 7443 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa674b3b50394f538d7ff773126ccf13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa674b3b50394f538d7ff773126ccf13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40217 } }
I20260807 11:28:42.857020 7443 leader_election.cc:302] T 00000000000000000000000000000000 P fa674b3b50394f538d7ff773126ccf13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa674b3b50394f538d7ff773126ccf13; no voters:
I20260807 11:28:42.857784 7443 leader_election.cc:288] T 00000000000000000000000000000000 P fa674b3b50394f538d7ff773126ccf13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:28:42.857899 7454 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa674b3b50394f538d7ff773126ccf13 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:42.859983 7443 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa674b3b50394f538d7ff773126ccf13 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:28:42.863736 7454 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa674b3b50394f538d7ff773126ccf13 [term 1 LEADER]: Becoming Leader. State: Replica: fa674b3b50394f538d7ff773126ccf13, State: Running, Role: LEADER
I20260807 11:28:42.864399 7454 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa674b3b50394f538d7ff773126ccf13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa674b3b50394f538d7ff773126ccf13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40217 } }
I20260807 11:28:42.867151 7455 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa674b3b50394f538d7ff773126ccf13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa674b3b50394f538d7ff773126ccf13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa674b3b50394f538d7ff773126ccf13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40217 } } }
I20260807 11:28:42.867571 7455 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa674b3b50394f538d7ff773126ccf13 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:42.868085 7456 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa674b3b50394f538d7ff773126ccf13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa674b3b50394f538d7ff773126ccf13. Latest consensus state: current_term: 1 leader_uuid: "fa674b3b50394f538d7ff773126ccf13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa674b3b50394f538d7ff773126ccf13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40217 } } }
I20260807 11:28:42.868366 7456 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa674b3b50394f538d7ff773126ccf13 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:42.876011 7462 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:28:42.881786 7447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:42.857187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:45473 (local address 127.0.0.1:40217)
0807 11:28:42.857378 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:42.857382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:42.858030 (+ 648us) server_negotiation.cc:407] Connection header received
0807 11:28:42.858292 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:42.858299 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:42.858404 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:42.858527 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:42.861263 (+ 2736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:42.863405 (+ 2142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:42.866320 (+ 2915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:42.867368 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:42.867515 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:42.872402 (+ 4887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:42.872413 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:42.872422 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:42.872482 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:42.877416 (+ 4934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:42.878662 (+ 1246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:42.878671 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:42.878677 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:42.878878 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:42.879738 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:42.879744 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:42.879751 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:42.880238 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:42.880563 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:42.881215 (+ 652us) server_negotiation.cc:299] Negotiation successful
0807 11:28:42.881426 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 11:28:42.883478 7462 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:28:42.893481 7462 catalog_manager.cc:1347] Generated new cluster ID: cfeaa1f24a924e8aafc92a1b273c0fb2
I20260807 11:28:42.893610 7462 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:28:42.902781 7462 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:28:42.904139 7462 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:28:42.913599 7462 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa674b3b50394f538d7ff773126ccf13: Generated new TSK 0
I20260807 11:28:42.914448 7462 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:28:42.922322 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 7373
I20260807 11:28:42.923590 7438 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:43.015451 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 7373
W20260807 11:28:43.035458 7357 connection.cc:535] client connection to 127.0.0.1:40217 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40217): Connection reset by peer (error 104)
2026-08-07T11:28:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2108 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:28:43 dist-test-slave-2w3w krb5kdc[7479](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:28:43 dist-test-slave-2w3w krb5kdc[7479](info): set up 2 sockets
Aug 07 11:28:43 dist-test-slave-2w3w krb5kdc[7479](info): commencing operation
krb5kdc: starting...
W20260807 11:28:44.154160 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:28:44 dist-test-slave-2w3w krb5kdc[7479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:28:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:28:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 11:28:44.488363 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37463
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37463
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:44.836437 7492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:44.836781 7492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:44.836898 7492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:44.844552 7492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:28:44.844682 7492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:44.844730 7492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:44.844841 7492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:28:44.844906 7492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:28:44.855034 7492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37463
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:44.856477 7492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:44.857898 7492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:44.870834 7497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:44.870872 7500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:44.870929 7498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:44.871821 7492 server_base.cc:1034] running on GCE node
I20260807 11:28:44.872710 7492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:44.874307 7492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:44.875528 7492 hybrid_clock.cc:648] HybridClock initialized: now 1786102124875458 us; error 79 us; skew 500 ppm
I20260807 11:28:44.875900 7492 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:44 dist-test-slave-2w3w krb5kdc[7479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102124, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:44.879333 7492 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 11:28:44.880944 7492 webserver.cc:460] Webserver started at http://127.0.0.1:41413/ using document root <none> and password file <none>
I20260807 11:28:44.881685 7492 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:44.881839 7492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:44.882153 7492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:44.885392 7492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "aae6262620a44dd191638443a7d1aa00"
format_stamp: "Formatted at 2026-08-07 11:28:44 on dist-test-slave-2w3w"
server_key: "662459483553b883b606db8e750fea6a"
server_key_iv: "aeb8edae3bfb66b96973f8652567dac0"
server_key_version: "encryptionkey@0"
I20260807 11:28:44.886214 7492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "aae6262620a44dd191638443a7d1aa00"
format_stamp: "Formatted at 2026-08-07 11:28:44 on dist-test-slave-2w3w"
server_key: "662459483553b883b606db8e750fea6a"
server_key_iv: "aeb8edae3bfb66b96973f8652567dac0"
server_key_version: "encryptionkey@0"
I20260807 11:28:44.891804 7492 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 11:28:44.895326 7507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:44.896700 7492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 11:28:44.896890 7492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "aae6262620a44dd191638443a7d1aa00"
format_stamp: "Formatted at 2026-08-07 11:28:44 on dist-test-slave-2w3w"
server_key: "662459483553b883b606db8e750fea6a"
server_key_iv: "aeb8edae3bfb66b96973f8652567dac0"
server_key_version: "encryptionkey@0"
I20260807 11:28:44.897116 7492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:44.919697 7492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:44.924237 7492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:44.924556 7492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:44.943636 7492 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37463
I20260807 11:28:44.943681 7559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37463 every 8 connection(s)
I20260807 11:28:44.945055 7492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:28:44.948881 7560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:44.949221 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 7492
I20260807 11:28:44.949421 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:28:44.949985 17172 external_mini_cluster.cc:1428] Setting key 4c0e73621f7992a99c2cf1a45f25c040
I20260807 11:28:44.962399 7560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aae6262620a44dd191638443a7d1aa00: Bootstrap starting.
I20260807 11:28:44.963441 7563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:44.951935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:60491 (local address 127.0.0.1:37463)
0807 11:28:44.952739 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:44.952750 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:44.952790 (+ 40us) server_negotiation.cc:407] Connection header received
0807 11:28:44.954294 (+ 1504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:44.954316 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:44.954743 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:44.955214 (+ 471us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:44.955802 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:44.957195 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:44.958540 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:44.959337 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:44.959563 (+ 226us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:44.960966 (+ 1403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:44.960973 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:44.960990 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:44.961559 (+ 569us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:44.961642 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:44.961668 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:44.961838 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:44.962139 (+ 301us) server_negotiation.cc:299] Negotiation successful
0807 11:28:44.962314 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":335,"threads_started":1}
I20260807 11:28:44.965822 7560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aae6262620a44dd191638443a7d1aa00: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:44.967146 7560 log.cc:824] T 00000000000000000000000000000000 P aae6262620a44dd191638443a7d1aa00: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:44.967428 17172 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 11:28:44.970224 7560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aae6262620a44dd191638443a7d1aa00: No bootstrap required, opened a new log
I20260807 11:28:44.978981 7563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:44.971864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:52343 (local address 127.0.0.1:37463)
0807 11:28:44.972065 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:44.972071 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:44.972626 (+ 555us) server_negotiation.cc:407] Connection header received
0807 11:28:44.972787 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:44.972792 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:44.972883 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:44.973004 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:44.973525 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:44.974703 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:44.976077 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:44.976828 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:44.976977 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:44.978089 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:44.978096 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:44.978101 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:44.978125 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:44.978200 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:44.978221 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:44.978346 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:44.978529 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 11:28:44.978597 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 11:28:44.980559 7560 raft_consensus.cc:348] T 00000000000000000000000000000000 P aae6262620a44dd191638443a7d1aa00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aae6262620a44dd191638443a7d1aa00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37463 } }
I20260807 11:28:44.981009 7560 raft_consensus.cc:374] T 00000000000000000000000000000000 P aae6262620a44dd191638443a7d1aa00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:44.981091 7560 raft_consensus.cc:729] T 00000000000000000000000000000000 P aae6262620a44dd191638443a7d1aa00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aae6262620a44dd191638443a7d1aa00, State: Initialized, Role: FOLLOWER
I20260807 11:28:44.981667 7560 consensus_queue.cc:260] T 00000000000000000000000000000000 P aae6262620a44dd191638443a7d1aa00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aae6262620a44dd191638443a7d1aa00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37463 } }
I20260807 11:28:44.981896 7560 raft_consensus.cc:388] T 00000000000000000000000000000000 P aae6262620a44dd191638443a7d1aa00 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:28:44.982000 7560 raft_consensus.cc:482] T 00000000000000000000000000000000 P aae6262620a44dd191638443a7d1aa00 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:28:44.982133 7560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aae6262620a44dd191638443a7d1aa00 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:44.984481 7560 raft_consensus.cc:504] T 00000000000000000000000000000000 P aae6262620a44dd191638443a7d1aa00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aae6262620a44dd191638443a7d1aa00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37463 } }
I20260807 11:28:44.985061 7560 leader_election.cc:302] T 00000000000000000000000000000000 P aae6262620a44dd191638443a7d1aa00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aae6262620a44dd191638443a7d1aa00; no voters:
I20260807 11:28:44.985651 7560 leader_election.cc:288] T 00000000000000000000000000000000 P aae6262620a44dd191638443a7d1aa00 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:28:44.985812 7570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aae6262620a44dd191638443a7d1aa00 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:44.987668 7570 raft_consensus.cc:686] T 00000000000000000000000000000000 P aae6262620a44dd191638443a7d1aa00 [term 1 LEADER]: Becoming Leader. State: Replica: aae6262620a44dd191638443a7d1aa00, State: Running, Role: LEADER
I20260807 11:28:44.988175 7560 sys_catalog.cc:564] T 00000000000000000000000000000000 P aae6262620a44dd191638443a7d1aa00 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:28:44.988281 7570 consensus_queue.cc:237] T 00000000000000000000000000000000 P aae6262620a44dd191638443a7d1aa00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aae6262620a44dd191638443a7d1aa00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37463 } }
I20260807 11:28:44.991294 7572 sys_catalog.cc:455] T 00000000000000000000000000000000 P aae6262620a44dd191638443a7d1aa00 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aae6262620a44dd191638443a7d1aa00. Latest consensus state: current_term: 1 leader_uuid: "aae6262620a44dd191638443a7d1aa00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aae6262620a44dd191638443a7d1aa00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37463 } } }
I20260807 11:28:44.991726 7571 sys_catalog.cc:455] T 00000000000000000000000000000000 P aae6262620a44dd191638443a7d1aa00 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aae6262620a44dd191638443a7d1aa00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aae6262620a44dd191638443a7d1aa00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37463 } } }
I20260807 11:28:44.992164 7571 sys_catalog.cc:458] T 00000000000000000000000000000000 P aae6262620a44dd191638443a7d1aa00 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:44.991760 7572 sys_catalog.cc:458] T 00000000000000000000000000000000 P aae6262620a44dd191638443a7d1aa00 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:44.993726 7579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:28:44.999233 7579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:28:45.009354 7579 catalog_manager.cc:1347] Generated new cluster ID: 215cf66fc3aa43b7b0b4036a3e6f65e4
I20260807 11:28:45.009507 7579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:28:45.029026 7579 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:28:45.030591 7579 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:28:45.039096 7579 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aae6262620a44dd191638443a7d1aa00: Generated new TSK 0
I20260807 11:28:45.040445 7579 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:28:45.061476 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 7492
W20260807 11:28:45.062732 7555 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.84:60491
W20260807 11:28:45.063783 17172 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 7492: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 11:28:45.063949 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 7492
W20260807 11:28:45.083354 7477 connection.cc:535] client connection to 127.0.0.1:37463 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37463): Connection reset by peer (error 104)
2026-08-07T11:28:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2049 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:28:45 dist-test-slave-2w3w krb5kdc[7592](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:28:45 dist-test-slave-2w3w krb5kdc[7592](info): set up 2 sockets
Aug 07 11:28:45 dist-test-slave-2w3w krb5kdc[7592](info): commencing operation
krb5kdc: starting...
W20260807 11:28:46.225667 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:28:46 dist-test-slave-2w3w krb5kdc[7592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:28:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:28:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 11:28:46.525913 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37477
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:46.870817 7605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:46.871176 7605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:46.871272 7605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:46.878743 7605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:28:46.878885 7605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:46.878942 7605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:46.879030 7605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:28:46.879077 7605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:28:46.889219 7605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37477
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786101765888975-17172-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:46.890542 7605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:46.891875 7605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:46.904026 7613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:46.904043 7611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:46.904385 7610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:46.908499 7605 server_base.cc:1034] running on GCE node
I20260807 11:28:46.909355 7605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:46.910885 7605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:46.912144 7605 hybrid_clock.cc:648] HybridClock initialized: now 1786102126912104 us; error 31 us; skew 500 ppm
I20260807 11:28:46.912528 7605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:46 dist-test-slave-2w3w krb5kdc[7592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102126, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:46.921211 7605 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 11:28:46.922919 7605 webserver.cc:460] Webserver started at http://127.0.0.1:39719/ using document root <none> and password file <none>
I20260807 11:28:46.923779 7605 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:46.923938 7605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:46.924263 7605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:46.927485 7605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "6968e03fa4364fc6a95cf27098d82b2d"
format_stamp: "Formatted at 2026-08-07 11:28:46 on dist-test-slave-2w3w"
server_key: "04496e44159dc2fd699a81bd60b27379"
server_key_iv: "7c387355ab6fc186a4618ebf8e08ad95"
server_key_version: "encryptionkey@0"
I20260807 11:28:46.928308 7605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "6968e03fa4364fc6a95cf27098d82b2d"
format_stamp: "Formatted at 2026-08-07 11:28:46 on dist-test-slave-2w3w"
server_key: "04496e44159dc2fd699a81bd60b27379"
server_key_iv: "7c387355ab6fc186a4618ebf8e08ad95"
server_key_version: "encryptionkey@0"
I20260807 11:28:46.933863 7605 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 11:28:46.937453 7620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:46.938848 7605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260807 11:28:46.939095 7605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "6968e03fa4364fc6a95cf27098d82b2d"
format_stamp: "Formatted at 2026-08-07 11:28:46 on dist-test-slave-2w3w"
server_key: "04496e44159dc2fd699a81bd60b27379"
server_key_iv: "7c387355ab6fc186a4618ebf8e08ad95"
server_key_version: "encryptionkey@0"
I20260807 11:28:46.939320 7605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:46.945218 7605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:46.949379 7605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:46.949699 7605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:46.969521 7605 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37477
I20260807 11:28:46.969564 7674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37477 every 8 connection(s)
I20260807 11:28:46.971086 7605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:28:46.975145 7675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:46.975188 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 7605
I20260807 11:28:46.975379 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:28:46.975987 17172 external_mini_cluster.cc:1428] Setting key 2e63446e3fb7e8d743b0ab974a985953
I20260807 11:28:46.983907 7678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:46.977874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:58665 (local address 127.0.0.1:37477)
0807 11:28:46.978801 (+ 927us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:46.978813 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:46.978855 (+ 42us) server_negotiation.cc:407] Connection header received
0807 11:28:46.980418 (+ 1563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:46.980441 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:46.980782 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:46.981123 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:46.982250 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:46.982256 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:46.982269 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:46.982285 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:46.982356 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:46.982379 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:46.982558 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:46.982671 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 11:28:46.982888 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":695,"thread_start_us":437,"threads_started":1}
I20260807 11:28:46.987811 17172 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 11:28:46.989492 7675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6968e03fa4364fc6a95cf27098d82b2d: Bootstrap starting.
I20260807 11:28:46.993697 7675 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6968e03fa4364fc6a95cf27098d82b2d: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:46.995054 7675 log.cc:824] T 00000000000000000000000000000000 P 6968e03fa4364fc6a95cf27098d82b2d: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:46.995940 7678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:46.993185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:45115 (local address 127.0.0.1:37477)
0807 11:28:46.993382 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:46.993388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:46.993794 (+ 406us) server_negotiation.cc:407] Connection header received
0807 11:28:46.994009 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:46.994015 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:46.994094 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:46.994233 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:46.995184 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:46.995189 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:46.995194 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:46.995213 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:46.995260 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:46.995281 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:46.995398 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:46.995523 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 11:28:46.995591 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 11:28:46.998185 7675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6968e03fa4364fc6a95cf27098d82b2d: No bootstrap required, opened a new log
I20260807 11:28:47.005927 7675 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6968e03fa4364fc6a95cf27098d82b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6968e03fa4364fc6a95cf27098d82b2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37477 } }
I20260807 11:28:47.006317 7675 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6968e03fa4364fc6a95cf27098d82b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:47.006445 7675 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6968e03fa4364fc6a95cf27098d82b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6968e03fa4364fc6a95cf27098d82b2d, State: Initialized, Role: FOLLOWER
I20260807 11:28:47.007097 7675 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6968e03fa4364fc6a95cf27098d82b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6968e03fa4364fc6a95cf27098d82b2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37477 } }
I20260807 11:28:47.007313 7675 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6968e03fa4364fc6a95cf27098d82b2d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:28:47.007478 7675 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6968e03fa4364fc6a95cf27098d82b2d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:28:47.007654 7675 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6968e03fa4364fc6a95cf27098d82b2d [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:47.009963 7675 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6968e03fa4364fc6a95cf27098d82b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6968e03fa4364fc6a95cf27098d82b2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37477 } }
I20260807 11:28:47.010607 7675 leader_election.cc:302] T 00000000000000000000000000000000 P 6968e03fa4364fc6a95cf27098d82b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6968e03fa4364fc6a95cf27098d82b2d; no voters:
I20260807 11:28:47.011335 7675 leader_election.cc:288] T 00000000000000000000000000000000 P 6968e03fa4364fc6a95cf27098d82b2d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:28:47.011476 7685 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6968e03fa4364fc6a95cf27098d82b2d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:47.012238 7685 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6968e03fa4364fc6a95cf27098d82b2d [term 1 LEADER]: Becoming Leader. State: Replica: 6968e03fa4364fc6a95cf27098d82b2d, State: Running, Role: LEADER
I20260807 11:28:47.012889 7685 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6968e03fa4364fc6a95cf27098d82b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6968e03fa4364fc6a95cf27098d82b2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37477 } }
I20260807 11:28:47.013584 7675 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6968e03fa4364fc6a95cf27098d82b2d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:28:47.016279 7686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6968e03fa4364fc6a95cf27098d82b2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6968e03fa4364fc6a95cf27098d82b2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6968e03fa4364fc6a95cf27098d82b2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37477 } } }
I20260807 11:28:47.016726 7686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6968e03fa4364fc6a95cf27098d82b2d [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:47.016688 7687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6968e03fa4364fc6a95cf27098d82b2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6968e03fa4364fc6a95cf27098d82b2d. Latest consensus state: current_term: 1 leader_uuid: "6968e03fa4364fc6a95cf27098d82b2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6968e03fa4364fc6a95cf27098d82b2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37477 } } }
I20260807 11:28:47.016991 7687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6968e03fa4364fc6a95cf27098d82b2d [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:47.018828 7695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:28:47.023836 7695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:28:47.032760 7695 catalog_manager.cc:1347] Generated new cluster ID: 90d083fef6634fc1bd16f35eeacbec3f
I20260807 11:28:47.032907 7695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:28:47.058087 7695 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:28:47.058547 7695 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:28:47.065330 7695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6968e03fa4364fc6a95cf27098d82b2d: Generated new TSK 0
I20260807 11:28:47.066290 7695 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:28:47.079308 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 7605
W20260807 11:28:47.080449 7670 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.84:58665
W20260807 11:28:47.081203 17172 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 7605: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 11:28:47.081365 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 7605
2026-08-07T11:28:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2014 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:28:47 dist-test-slave-2w3w krb5kdc[7708](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:28:47 dist-test-slave-2w3w krb5kdc[7708](info): set up 2 sockets
Aug 07 11:28:47 dist-test-slave-2w3w krb5kdc[7708](info): commencing operation
krb5kdc: starting...
W20260807 11:28:48.245321 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:28:48 dist-test-slave-2w3w krb5kdc[7708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:28:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:28:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:28:48.542606 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:33505
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:36547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:33505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:48.881836 7721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:48.882150 7721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:48.882328 7721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:48.889614 7721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:28:48.889734 7721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:48.889878 7721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:48.889987 7721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:28:48.890053 7721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:28:48.900161 7721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:36547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:33505
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:33505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:48.901554 7721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:48.903015 7721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:48.915585 7730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:48.915616 7727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:48.915885 7726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:48.916780 7721 server_base.cc:1034] running on GCE node
I20260807 11:28:48.917567 7721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:48.919015 7721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:48.920228 7721 hybrid_clock.cc:648] HybridClock initialized: now 1786102128920192 us; error 34 us; skew 500 ppm
I20260807 11:28:48.920555 7721 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:48 dist-test-slave-2w3w krb5kdc[7708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102128, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:48.923745 7721 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:28:48.925321 7721 webserver.cc:460] Webserver started at http://127.16.197.62:46325/ using document root <none> and password file <none>
I20260807 11:28:48.926013 7721 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:48.926186 7721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:48.926501 7721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:48.929626 7721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "4f6a0f04e957426dbd7f2d108af6cfe2"
format_stamp: "Formatted at 2026-08-07 11:28:48 on dist-test-slave-2w3w"
server_key: "04a0ce446e3446fef57c5c18fbc2a530"
server_key_iv: "f6b801db358ed61d38516ed91b27cd30"
server_key_version: "encryptionkey@0"
I20260807 11:28:48.930438 7721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "4f6a0f04e957426dbd7f2d108af6cfe2"
format_stamp: "Formatted at 2026-08-07 11:28:48 on dist-test-slave-2w3w"
server_key: "04a0ce446e3446fef57c5c18fbc2a530"
server_key_iv: "f6b801db358ed61d38516ed91b27cd30"
server_key_version: "encryptionkey@0"
I20260807 11:28:48.935762 7721 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 11:28:48.939237 7737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:48.940548 7721 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 11:28:48.940740 7721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "4f6a0f04e957426dbd7f2d108af6cfe2"
format_stamp: "Formatted at 2026-08-07 11:28:48 on dist-test-slave-2w3w"
server_key: "04a0ce446e3446fef57c5c18fbc2a530"
server_key_iv: "f6b801db358ed61d38516ed91b27cd30"
server_key_version: "encryptionkey@0"
I20260807 11:28:48.940956 7721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:48.964052 7721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:48.968123 7721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:48.968418 7721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:48.987601 7721 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:33505
I20260807 11:28:48.987634 7790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:33505 every 8 connection(s)
I20260807 11:28:48.989027 7721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:28:48.991514 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 7721
I20260807 11:28:48.991714 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:28:48.992282 17172 external_mini_cluster.cc:1428] Setting key 2e8ae46e441e6cd4df567632d1e88f1a
I20260807 11:28:48.993418 7791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 11:28:49 dist-test-slave-2w3w krb5kdc[7708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:28:49.008175 7791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f6a0f04e957426dbd7f2d108af6cfe2: Bootstrap starting.
I20260807 11:28:49.011901 7791 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f6a0f04e957426dbd7f2d108af6cfe2: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:49.012945 7791 log.cc:824] T 00000000000000000000000000000000 P 4f6a0f04e957426dbd7f2d108af6cfe2: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:49.016531 7791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f6a0f04e957426dbd7f2d108af6cfe2: No bootstrap required, opened a new log
I20260807 11:28:49.016582 7794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:48.994067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56851 (local address 127.16.197.62:33505)
0807 11:28:48.994841 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:48.994852 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:48.994893 (+ 41us) server_negotiation.cc:407] Connection header received
0807 11:28:48.996532 (+ 1639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:48.996554 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:48.996975 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:48.997312 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:48.999152 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:49.000508 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:49.002364 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:49.003111 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:49.003231 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:49.008270 (+ 5039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:49.008278 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:49.008293 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:49.008368 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:49.011630 (+ 3262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:49.012520 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:49.012527 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:49.012534 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:49.012706 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:49.013221 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:49.013226 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:49.013229 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:49.013937 (+ 708us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:49.014195 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:49.014838 (+ 643us) server_negotiation.cc:299] Negotiation successful
0807 11:28:49.015243 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":352,"threads_started":1}
I20260807 11:28:49.020412 17172 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 11:28:49.024845 7791 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f6a0f04e957426dbd7f2d108af6cfe2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f6a0f04e957426dbd7f2d108af6cfe2" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33505 } }
I20260807 11:28:49.025323 7791 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f6a0f04e957426dbd7f2d108af6cfe2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:49.025453 7791 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f6a0f04e957426dbd7f2d108af6cfe2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f6a0f04e957426dbd7f2d108af6cfe2, State: Initialized, Role: FOLLOWER
I20260807 11:28:49.026121 7791 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f6a0f04e957426dbd7f2d108af6cfe2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f6a0f04e957426dbd7f2d108af6cfe2" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33505 } }
I20260807 11:28:49.026388 7791 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f6a0f04e957426dbd7f2d108af6cfe2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:28:49.026532 7791 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f6a0f04e957426dbd7f2d108af6cfe2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:28:49.026721 7791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f6a0f04e957426dbd7f2d108af6cfe2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:49.029631 7791 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f6a0f04e957426dbd7f2d108af6cfe2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f6a0f04e957426dbd7f2d108af6cfe2" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33505 } }
I20260807 11:28:49.030252 7791 leader_election.cc:302] T 00000000000000000000000000000000 P 4f6a0f04e957426dbd7f2d108af6cfe2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f6a0f04e957426dbd7f2d108af6cfe2; no voters:
I20260807 11:28:49.030997 7791 leader_election.cc:288] T 00000000000000000000000000000000 P 4f6a0f04e957426dbd7f2d108af6cfe2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:28:49.031139 7801 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f6a0f04e957426dbd7f2d108af6cfe2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:49.033509 7791 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f6a0f04e957426dbd7f2d108af6cfe2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:28:49.035190 7801 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f6a0f04e957426dbd7f2d108af6cfe2 [term 1 LEADER]: Becoming Leader. State: Replica: 4f6a0f04e957426dbd7f2d108af6cfe2, State: Running, Role: LEADER
I20260807 11:28:49.035857 7801 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f6a0f04e957426dbd7f2d108af6cfe2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f6a0f04e957426dbd7f2d108af6cfe2" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33505 } }
I20260807 11:28:49.039268 7803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f6a0f04e957426dbd7f2d108af6cfe2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f6a0f04e957426dbd7f2d108af6cfe2. Latest consensus state: current_term: 1 leader_uuid: "4f6a0f04e957426dbd7f2d108af6cfe2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f6a0f04e957426dbd7f2d108af6cfe2" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33505 } } }
I20260807 11:28:49.039724 7803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f6a0f04e957426dbd7f2d108af6cfe2 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:49.040952 7802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f6a0f04e957426dbd7f2d108af6cfe2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f6a0f04e957426dbd7f2d108af6cfe2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f6a0f04e957426dbd7f2d108af6cfe2" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33505 } } }
I20260807 11:28:49.043009 7802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f6a0f04e957426dbd7f2d108af6cfe2 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:49.047533 7812 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:28:49.049535 7794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:49.024847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33745 (local address 127.16.197.62:33505)
0807 11:28:49.025036 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:49.025043 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:49.025452 (+ 409us) server_negotiation.cc:407] Connection header received
0807 11:28:49.025656 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:49.025662 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:49.025741 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:49.025865 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:49.027566 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:49.028751 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:49.029939 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:49.030616 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:49.030770 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:49.034826 (+ 4056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:49.034836 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:49.034844 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:49.034902 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:49.043750 (+ 8848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:49.045808 (+ 2058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:49.045817 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:49.045823 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:49.046324 (+ 501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:49.047408 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:49.047414 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:49.047417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:49.047866 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:49.048171 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:49.048856 (+ 685us) server_negotiation.cc:299] Negotiation successful
0807 11:28:49.049141 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 11:28:49.055970 7812 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:28:49.066115 7812 catalog_manager.cc:1347] Generated new cluster ID: 0526737c49bc4030b2ad35565996b473
I20260807 11:28:49.066243 7812 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:28:49.093008 7812 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:28:49.094393 7812 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:28:49.103452 7812 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f6a0f04e957426dbd7f2d108af6cfe2: Generated new TSK 0
I20260807 11:28:49.104353 7812 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:28:49.113205 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 7721
I20260807 11:28:49.115044 7786 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:49.211175 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 7721
W20260807 11:28:49.232702 7706 connection.cc:535] client connection to 127.16.197.62:33505 recv error: Network error: failed to read from TLS socket (remote: 127.16.197.62:33505): Connection reset by peer (error 104)
2026-08-07T11:28:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2131 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:28:49 dist-test-slave-2w3w krb5kdc[7823](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:28:49 dist-test-slave-2w3w krb5kdc[7823](info): set up 2 sockets
Aug 07 11:28:49 dist-test-slave-2w3w krb5kdc[7823](info): commencing operation
krb5kdc: starting...
W20260807 11:28:50.365216 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:28:50 dist-test-slave-2w3w krb5kdc[7823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:28:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:28:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:28:50.658010 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:39789
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:46015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:39789
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:51.002420 7837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:51.002782 7837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:51.003072 7837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:51.010456 7837 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:28:51.010620 7837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:51.010735 7837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:51.010846 7837 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:28:51.010939 7837 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:28:51.021051 7837 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:46015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:39789
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:39789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:51.022398 7837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:51.023802 7837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:51.036173 7843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:51.036598 7842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:51.036646 7845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:51.037163 7837 server_base.cc:1034] running on GCE node
I20260807 11:28:51.037904 7837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:51.039371 7837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:51.040577 7837 hybrid_clock.cc:648] HybridClock initialized: now 1786102131040517 us; error 53 us; skew 500 ppm
I20260807 11:28:51.040951 7837 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:51 dist-test-slave-2w3w krb5kdc[7823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102131, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:51.044270 7837 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:28:51.046487 7837 webserver.cc:460] Webserver started at http://127.16.197.62:45655/ using document root <none> and password file <none>
I20260807 11:28:51.047582 7837 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:51.047748 7837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:51.048189 7837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:51.052539 7837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "95bf2781177d4e39a0220e44cbbf7df0"
format_stamp: "Formatted at 2026-08-07 11:28:51 on dist-test-slave-2w3w"
server_key: "64d6da77b66f54b25698db9bd4c24c2f"
server_key_iv: "146cd87fb5e77d3c1613a4ca8fb0c4b8"
server_key_version: "encryptionkey@0"
I20260807 11:28:51.053367 7837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "95bf2781177d4e39a0220e44cbbf7df0"
format_stamp: "Formatted at 2026-08-07 11:28:51 on dist-test-slave-2w3w"
server_key: "64d6da77b66f54b25698db9bd4c24c2f"
server_key_iv: "146cd87fb5e77d3c1613a4ca8fb0c4b8"
server_key_version: "encryptionkey@0"
I20260807 11:28:51.058889 7837 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 11:28:51.062471 7854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:51.063866 7837 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:28:51.064056 7837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "95bf2781177d4e39a0220e44cbbf7df0"
format_stamp: "Formatted at 2026-08-07 11:28:51 on dist-test-slave-2w3w"
server_key: "64d6da77b66f54b25698db9bd4c24c2f"
server_key_iv: "146cd87fb5e77d3c1613a4ca8fb0c4b8"
server_key_version: "encryptionkey@0"
I20260807 11:28:51.064273 7837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:51.086872 7837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:51.091301 7837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:51.091622 7837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:51.111315 7837 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:39789
I20260807 11:28:51.111339 7906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:39789 every 8 connection(s)
I20260807 11:28:51.112746 7837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:28:51.116586 7907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:51.117131 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 7837
I20260807 11:28:51.117318 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:28:51.117914 17172 external_mini_cluster.cc:1428] Setting key 4efcf05d9c457e987cb2f1b1fee86605
I20260807 11:28:51.130841 7907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95bf2781177d4e39a0220e44cbbf7df0: Bootstrap starting.
I20260807 11:28:51.132434 7910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:51.119840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53119 (local address 127.16.197.62:39789)
0807 11:28:51.120675 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:51.120688 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:51.120730 (+ 42us) server_negotiation.cc:407] Connection header received
0807 11:28:51.123086 (+ 2356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:51.123110 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:51.123544 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:51.123882 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:51.124515 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:51.125768 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:51.127754 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:51.128464 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:51.128590 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:51.129951 (+ 1361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:51.129958 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:51.129971 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:51.130567 (+ 596us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:51.130640 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:51.130664 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:51.130788 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:51.131187 (+ 399us) server_negotiation.cc:299] Negotiation successful
0807 11:28:51.131382 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":392,"threads_started":1}
I20260807 11:28:51.134389 7907 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95bf2781177d4e39a0220e44cbbf7df0: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:51.135746 7907 log.cc:824] T 00000000000000000000000000000000 P 95bf2781177d4e39a0220e44cbbf7df0: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:51.136411 17172 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 11:28:51.138614 7907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95bf2781177d4e39a0220e44cbbf7df0: No bootstrap required, opened a new log
I20260807 11:28:51.147626 7910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:51.140710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41157 (local address 127.16.197.62:39789)
0807 11:28:51.140906 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:51.140912 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:51.141450 (+ 538us) server_negotiation.cc:407] Connection header received
0807 11:28:51.141600 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:51.141604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:51.141678 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:51.141782 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:51.142267 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:51.143454 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:51.144787 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:51.145517 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:51.145642 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:51.146796 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:51.146803 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:51.146808 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:51.146834 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:51.146907 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:51.146929 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:51.147063 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:51.147237 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 11:28:51.147298 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 11:28:51.148366 7907 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95bf2781177d4e39a0220e44cbbf7df0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95bf2781177d4e39a0220e44cbbf7df0" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 39789 } }
I20260807 11:28:51.148824 7907 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95bf2781177d4e39a0220e44cbbf7df0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:51.148962 7907 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95bf2781177d4e39a0220e44cbbf7df0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95bf2781177d4e39a0220e44cbbf7df0, State: Initialized, Role: FOLLOWER
I20260807 11:28:51.149632 7907 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95bf2781177d4e39a0220e44cbbf7df0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95bf2781177d4e39a0220e44cbbf7df0" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 39789 } }
I20260807 11:28:51.149894 7907 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95bf2781177d4e39a0220e44cbbf7df0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:28:51.150048 7907 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95bf2781177d4e39a0220e44cbbf7df0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:28:51.150225 7907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95bf2781177d4e39a0220e44cbbf7df0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:51.153007 7907 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95bf2781177d4e39a0220e44cbbf7df0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95bf2781177d4e39a0220e44cbbf7df0" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 39789 } }
I20260807 11:28:51.153668 7907 leader_election.cc:302] T 00000000000000000000000000000000 P 95bf2781177d4e39a0220e44cbbf7df0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95bf2781177d4e39a0220e44cbbf7df0; no voters:
I20260807 11:28:51.154438 7907 leader_election.cc:288] T 00000000000000000000000000000000 P 95bf2781177d4e39a0220e44cbbf7df0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:28:51.154554 7917 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95bf2781177d4e39a0220e44cbbf7df0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:51.156455 7917 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95bf2781177d4e39a0220e44cbbf7df0 [term 1 LEADER]: Becoming Leader. State: Replica: 95bf2781177d4e39a0220e44cbbf7df0, State: Running, Role: LEADER
I20260807 11:28:51.157073 7917 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95bf2781177d4e39a0220e44cbbf7df0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95bf2781177d4e39a0220e44cbbf7df0" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 39789 } }
I20260807 11:28:51.159039 7917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95bf2781177d4e39a0220e44cbbf7df0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95bf2781177d4e39a0220e44cbbf7df0. Latest consensus state: current_term: 1 leader_uuid: "95bf2781177d4e39a0220e44cbbf7df0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95bf2781177d4e39a0220e44cbbf7df0" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 39789 } } }
I20260807 11:28:51.159340 7917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95bf2781177d4e39a0220e44cbbf7df0 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:51.161510 7919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95bf2781177d4e39a0220e44cbbf7df0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95bf2781177d4e39a0220e44cbbf7df0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95bf2781177d4e39a0220e44cbbf7df0" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 39789 } } }
I20260807 11:28:51.161945 7919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95bf2781177d4e39a0220e44cbbf7df0 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:51.163435 7907 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95bf2781177d4e39a0220e44cbbf7df0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:28:51.163892 7921 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:28:51.170073 7921 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:28:51.180209 7921 catalog_manager.cc:1347] Generated new cluster ID: 04390be84c004e8b868d50c42f1fe5ad
I20260807 11:28:51.180349 7921 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:28:51.196051 7921 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:28:51.197407 7921 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:28:51.203876 7921 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95bf2781177d4e39a0220e44cbbf7df0: Generated new TSK 0
I20260807 11:28:51.204753 7921 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:28:51.221302 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 7837
W20260807 11:28:51.222942 7902 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53119
W20260807 11:28:51.223906 17172 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 7837: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 11:28:51.224069 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 7837
W20260807 11:28:51.252338 7821 connection.cc:535] client connection to 127.16.197.62:39789 recv error: Network error: failed to read from TLS socket (remote: 127.16.197.62:39789): Connection reset by peer (error 104)
2026-08-07T11:28:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2021 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:28:51 dist-test-slave-2w3w krb5kdc[7938](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:28:51 dist-test-slave-2w3w krb5kdc[7938](info): set up 2 sockets
Aug 07 11:28:51 dist-test-slave-2w3w krb5kdc[7938](info): commencing operation
krb5kdc: starting...
W20260807 11:28:52.413461 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:28:52 dist-test-slave-2w3w krb5kdc[7938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:28:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:28:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:28:52.696569 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:37765
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:43343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:37765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:53.045096 7953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:53.045589 7953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:53.045758 7953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:53.056856 7953 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:28:53.057037 7953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:53.057150 7953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:53.057247 7953 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:28:53.057319 7953 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:28:53.067898 7953 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:43343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:37765
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:37765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:53.069347 7953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:53.070748 7953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:53.083058 7958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:53.083015 7959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:53.083101 7961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:53.083897 7953 server_base.cc:1034] running on GCE node
I20260807 11:28:53.084734 7953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:53.086212 7953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:53.087455 7953 hybrid_clock.cc:648] HybridClock initialized: now 1786102133087420 us; error 57 us; skew 500 ppm
I20260807 11:28:53.087852 7953 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:53 dist-test-slave-2w3w krb5kdc[7938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102133, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:53.097497 7953 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:28:53.099253 7953 webserver.cc:460] Webserver started at http://127.16.197.62:41635/ using document root <none> and password file <none>
I20260807 11:28:53.100013 7953 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:53.100173 7953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:53.100503 7953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:53.103947 7953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "5c259df1999841fd9c4c236db404521c"
format_stamp: "Formatted at 2026-08-07 11:28:53 on dist-test-slave-2w3w"
server_key: "ab42270c3b0e2868eb1945456cc91e78"
server_key_iv: "000c5c000796a4a1560e12d1062c675f"
server_key_version: "encryptionkey@0"
I20260807 11:28:53.104755 7953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "5c259df1999841fd9c4c236db404521c"
format_stamp: "Formatted at 2026-08-07 11:28:53 on dist-test-slave-2w3w"
server_key: "ab42270c3b0e2868eb1945456cc91e78"
server_key_iv: "000c5c000796a4a1560e12d1062c675f"
server_key_version: "encryptionkey@0"
I20260807 11:28:53.110121 7953 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 11:28:53.113685 7968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:53.115163 7953 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:28:53.115361 7953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "5c259df1999841fd9c4c236db404521c"
format_stamp: "Formatted at 2026-08-07 11:28:53 on dist-test-slave-2w3w"
server_key: "ab42270c3b0e2868eb1945456cc91e78"
server_key_iv: "000c5c000796a4a1560e12d1062c675f"
server_key_version: "encryptionkey@0"
I20260807 11:28:53.115576 7953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:53.121672 7953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:53.126339 7953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:53.126667 7953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:53.147270 7953 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:37765
I20260807 11:28:53.147306 8021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:37765 every 8 connection(s)
I20260807 11:28:53.148844 7953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:28:53.152918 8022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:53.155628 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 7953
I20260807 11:28:53.155830 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:28:53.156387 17172 external_mini_cluster.cc:1428] Setting key 81680d2611240242c1336f6f46e33452
I20260807 11:28:53.166363 8025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:53.158420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53811 (local address 127.16.197.62:37765)
0807 11:28:53.159375 (+ 955us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:53.159388 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:53.159433 (+ 45us) server_negotiation.cc:407] Connection header received
0807 11:28:53.162132 (+ 2699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:53.162187 (+ 55us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:53.162786 (+ 599us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:53.163326 (+ 540us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:53.164723 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:53.164732 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:53.164745 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:53.164757 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:53.164809 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:53.164827 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:53.165078 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:53.165213 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 11:28:53.165419 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":398,"threads_started":1}
I20260807 11:28:53.166739 8022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c259df1999841fd9c4c236db404521c: Bootstrap starting.
I20260807 11:28:53.170161 8022 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c259df1999841fd9c4c236db404521c: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:53.170164 17172 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 11:28:53.171231 8022 log.cc:824] T 00000000000000000000000000000000 P 5c259df1999841fd9c4c236db404521c: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:53.174378 8022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c259df1999841fd9c4c236db404521c: No bootstrap required, opened a new log
I20260807 11:28:53.177709 8025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:53.175188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33499 (local address 127.16.197.62:37765)
0807 11:28:53.175422 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:53.175428 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:53.175763 (+ 335us) server_negotiation.cc:407] Connection header received
0807 11:28:53.175940 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:53.175944 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:53.175991 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:53.176071 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:53.177036 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:53.177040 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:53.177043 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:53.177057 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:53.177094 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:53.177108 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:53.177203 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:53.177343 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 11:28:53.177389 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 11:28:53.183421 8022 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c259df1999841fd9c4c236db404521c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c259df1999841fd9c4c236db404521c" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 37765 } }
I20260807 11:28:53.183805 8022 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c259df1999841fd9c4c236db404521c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:53.183923 8022 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c259df1999841fd9c4c236db404521c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c259df1999841fd9c4c236db404521c, State: Initialized, Role: FOLLOWER
I20260807 11:28:53.184511 8022 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c259df1999841fd9c4c236db404521c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c259df1999841fd9c4c236db404521c" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 37765 } }
I20260807 11:28:53.184762 8022 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c259df1999841fd9c4c236db404521c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:28:53.184913 8022 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c259df1999841fd9c4c236db404521c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:28:53.185101 8022 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c259df1999841fd9c4c236db404521c [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:53.187839 8022 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c259df1999841fd9c4c236db404521c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c259df1999841fd9c4c236db404521c" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 37765 } }
I20260807 11:28:53.188349 8022 leader_election.cc:302] T 00000000000000000000000000000000 P 5c259df1999841fd9c4c236db404521c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c259df1999841fd9c4c236db404521c; no voters:
I20260807 11:28:53.188954 8022 leader_election.cc:288] T 00000000000000000000000000000000 P 5c259df1999841fd9c4c236db404521c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:28:53.189102 8033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c259df1999841fd9c4c236db404521c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:53.189786 8033 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c259df1999841fd9c4c236db404521c [term 1 LEADER]: Becoming Leader. State: Replica: 5c259df1999841fd9c4c236db404521c, State: Running, Role: LEADER
I20260807 11:28:53.190371 8033 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c259df1999841fd9c4c236db404521c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c259df1999841fd9c4c236db404521c" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 37765 } }
I20260807 11:28:53.191139 8022 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c259df1999841fd9c4c236db404521c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:28:53.193339 8034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c259df1999841fd9c4c236db404521c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c259df1999841fd9c4c236db404521c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c259df1999841fd9c4c236db404521c" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 37765 } } }
I20260807 11:28:53.193691 8034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c259df1999841fd9c4c236db404521c [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:53.195684 8035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c259df1999841fd9c4c236db404521c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c259df1999841fd9c4c236db404521c. Latest consensus state: current_term: 1 leader_uuid: "5c259df1999841fd9c4c236db404521c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c259df1999841fd9c4c236db404521c" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 37765 } } }
I20260807 11:28:53.195997 8035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c259df1999841fd9c4c236db404521c [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:53.200544 8040 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:28:53.206364 8040 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:28:53.218938 8040 catalog_manager.cc:1347] Generated new cluster ID: e627b39eaed4459894e1292e30b5d4df
I20260807 11:28:53.219152 8040 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:28:53.254136 8040 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:28:53.254583 8040 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:28:53.262436 8040 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c259df1999841fd9c4c236db404521c: Generated new TSK 0
I20260807 11:28:53.263546 8040 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:28:53.280537 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 7953
W20260807 11:28:53.281921 8017 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53811
W20260807 11:28:53.282702 17172 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 7953: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 11:28:53.282866 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 7953
2026-08-07T11:28:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2047 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:28:53 dist-test-slave-2w3w krb5kdc[8055](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:28:53 dist-test-slave-2w3w krb5kdc[8055](info): set up 2 sockets
Aug 07 11:28:53 dist-test-slave-2w3w krb5kdc[8055](info): commencing operation
krb5kdc: starting...
W20260807 11:28:54.437482 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.004s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:28:54 dist-test-slave-2w3w krb5kdc[8055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:28:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:28:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:28:54.755434 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:46101
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:35771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:46101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:55.120504 8068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:55.120903 8068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:55.121078 8068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:55.128648 8068 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:28:55.128829 8068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:55.128942 8068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:55.129057 8068 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:28:55.129135 8068 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:28:55.139716 8068 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:35771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:46101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:46101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:55.141180 8068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:55.142587 8068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:55.155529 8076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:55.155752 8074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:55.156095 8078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:55.156602 8068 server_base.cc:1034] running on GCE node
I20260807 11:28:55.157480 8068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:55.159014 8068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:55.160261 8068 hybrid_clock.cc:648] HybridClock initialized: now 1786102135160213 us; error 54 us; skew 500 ppm
I20260807 11:28:55.160636 8068 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:55 dist-test-slave-2w3w krb5kdc[8055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102135, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:55.163780 8068 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:28:55.165277 8068 webserver.cc:460] Webserver started at http://127.16.197.62:41981/ using document root <none> and password file <none>
I20260807 11:28:55.165982 8068 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:55.166128 8068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:55.166445 8068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:55.170347 8068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "3813f099d70b428388098803d879e400"
format_stamp: "Formatted at 2026-08-07 11:28:55 on dist-test-slave-2w3w"
server_key: "e88b0e5375a9407efb30167eb12332e4"
server_key_iv: "06a06cc7face43594bfe067cefd45553"
server_key_version: "encryptionkey@0"
I20260807 11:28:55.171377 8068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "3813f099d70b428388098803d879e400"
format_stamp: "Formatted at 2026-08-07 11:28:55 on dist-test-slave-2w3w"
server_key: "e88b0e5375a9407efb30167eb12332e4"
server_key_iv: "06a06cc7face43594bfe067cefd45553"
server_key_version: "encryptionkey@0"
I20260807 11:28:55.176815 8068 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 11:28:55.180280 8085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:55.181609 8068 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 11:28:55.181806 8068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "3813f099d70b428388098803d879e400"
format_stamp: "Formatted at 2026-08-07 11:28:55 on dist-test-slave-2w3w"
server_key: "e88b0e5375a9407efb30167eb12332e4"
server_key_iv: "06a06cc7face43594bfe067cefd45553"
server_key_version: "encryptionkey@0"
I20260807 11:28:55.182025 8068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:55.199664 8068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:55.203703 8068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:55.204016 8068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:55.223441 8068 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:46101
I20260807 11:28:55.223466 8139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:46101 every 8 connection(s)
I20260807 11:28:55.224906 8068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:28:55.226891 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 8068
I20260807 11:28:55.227106 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:28:55.227686 17172 external_mini_cluster.cc:1428] Setting key c2a124795f836a54d11a3c549b0918ce
I20260807 11:28:55.229346 8140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:55.241266 8140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3813f099d70b428388098803d879e400: Bootstrap starting.
Aug 07 11:28:55 dist-test-slave-2w3w krb5kdc[8055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:28:55.245313 8140 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3813f099d70b428388098803d879e400: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:55.246522 8140 log.cc:824] T 00000000000000000000000000000000 P 3813f099d70b428388098803d879e400: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:55.250092 8140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3813f099d70b428388098803d879e400: No bootstrap required, opened a new log
I20260807 11:28:55.251581 8143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:55.229608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54377 (local address 127.16.197.62:46101)
0807 11:28:55.230368 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:55.230380 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:55.230420 (+ 40us) server_negotiation.cc:407] Connection header received
0807 11:28:55.232129 (+ 1709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:55.232151 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:55.232548 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:55.232958 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:55.234403 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:55.235825 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:55.237102 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:55.237952 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:55.238162 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:55.243155 (+ 4993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:55.243169 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:55.243187 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:55.243239 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:55.247037 (+ 3798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:55.247903 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:55.247914 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:55.247930 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:55.248119 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:55.248569 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:55.248572 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:55.248574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:55.249126 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:55.249351 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:55.249899 (+ 548us) server_negotiation.cc:299] Negotiation successful
0807 11:28:55.250474 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":332,"threads_started":1}
I20260807 11:28:55.255695 17172 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 11:28:55.259995 8140 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3813f099d70b428388098803d879e400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3813f099d70b428388098803d879e400" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46101 } }
I20260807 11:28:55.260504 8140 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3813f099d70b428388098803d879e400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:55.260620 8140 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3813f099d70b428388098803d879e400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3813f099d70b428388098803d879e400, State: Initialized, Role: FOLLOWER
I20260807 11:28:55.261370 8140 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3813f099d70b428388098803d879e400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3813f099d70b428388098803d879e400" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46101 } }
I20260807 11:28:55.261683 8140 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3813f099d70b428388098803d879e400 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:28:55.261868 8140 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3813f099d70b428388098803d879e400 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:28:55.262084 8140 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3813f099d70b428388098803d879e400 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:55.264629 8140 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3813f099d70b428388098803d879e400 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3813f099d70b428388098803d879e400" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46101 } }
I20260807 11:28:55.265221 8140 leader_election.cc:302] T 00000000000000000000000000000000 P 3813f099d70b428388098803d879e400 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3813f099d70b428388098803d879e400; no voters:
I20260807 11:28:55.265990 8140 leader_election.cc:288] T 00000000000000000000000000000000 P 3813f099d70b428388098803d879e400 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:28:55.266104 8151 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3813f099d70b428388098803d879e400 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:55.268821 8140 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3813f099d70b428388098803d879e400 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:28:55.272140 8151 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3813f099d70b428388098803d879e400 [term 1 LEADER]: Becoming Leader. State: Replica: 3813f099d70b428388098803d879e400, State: Running, Role: LEADER
I20260807 11:28:55.272991 8151 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3813f099d70b428388098803d879e400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3813f099d70b428388098803d879e400" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46101 } }
I20260807 11:28:55.278877 8143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:55.261264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60179 (local address 127.16.197.62:46101)
0807 11:28:55.261481 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:55.261488 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:55.262044 (+ 556us) server_negotiation.cc:407] Connection header received
0807 11:28:55.262261 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:55.262266 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:55.262356 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:55.262477 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:55.264090 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:55.264880 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:55.266277 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:55.267011 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:55.267126 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:55.269918 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:55.269924 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:55.269930 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:28:55.269978 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:55.274023 (+ 4045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:55.275139 (+ 1116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:55.275148 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:55.275154 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:55.275381 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:28:55.275973 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:28:55.275981 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:28:55.275986 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:28:55.276493 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:55.276828 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:55.277988 (+ 1160us) server_negotiation.cc:299] Negotiation successful
0807 11:28:55.278254 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 11:28:55.279047 8152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3813f099d70b428388098803d879e400 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3813f099d70b428388098803d879e400" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3813f099d70b428388098803d879e400" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46101 } } }
I20260807 11:28:55.279457 8152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3813f099d70b428388098803d879e400 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:55.281934 8153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3813f099d70b428388098803d879e400 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3813f099d70b428388098803d879e400. Latest consensus state: current_term: 1 leader_uuid: "3813f099d70b428388098803d879e400" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3813f099d70b428388098803d879e400" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 46101 } } }
I20260807 11:28:55.282327 8153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3813f099d70b428388098803d879e400 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:55.283689 8162 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:28:55.289021 8162 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:28:55.297477 8162 catalog_manager.cc:1347] Generated new cluster ID: 33207aa00fba4431905a2fb2edc8c6bf
I20260807 11:28:55.297668 8162 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:28:55.307950 8162 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:28:55.309427 8162 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:28:55.319233 8162 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3813f099d70b428388098803d879e400: Generated new TSK 0
I20260807 11:28:55.320171 8162 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:28:55.331931 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 8068
I20260807 11:28:55.333227 8135 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:28:55.426340 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 8068
2026-08-07T11:28:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2140 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:28:55 dist-test-slave-2w3w krb5kdc[8174](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:28:55 dist-test-slave-2w3w krb5kdc[8174](info): set up 2 sockets
Aug 07 11:28:55 dist-test-slave-2w3w krb5kdc[8174](info): commencing operation
krb5kdc: starting...
W20260807 11:28:56.593170 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:28:56 dist-test-slave-2w3w krb5kdc[8174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:28:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:28:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:28:56.906363 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:33273
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:38437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:33273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:57.264837 8187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:57.265183 8187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:57.265370 8187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:57.272931 8187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:28:57.273063 8187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:57.273186 8187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:57.273293 8187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:28:57.273376 8187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:28:57.283710 8187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:38437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:33273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:33273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:57.285136 8187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:57.286556 8187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:57.299293 8192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:57.299192 8195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:57.299232 8193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:57.300158 8187 server_base.cc:1034] running on GCE node
I20260807 11:28:57.301056 8187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:57.302513 8187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:57.303715 8187 hybrid_clock.cc:648] HybridClock initialized: now 1786102137303682 us; error 30 us; skew 500 ppm
I20260807 11:28:57.304046 8187 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:57 dist-test-slave-2w3w krb5kdc[8174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102137, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:57.307384 8187 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:28:57.308930 8187 webserver.cc:460] Webserver started at http://127.16.197.62:43229/ using document root <none> and password file <none>
I20260807 11:28:57.309659 8187 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:57.309803 8187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:57.310452 8187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:57.313791 8187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "e09789d199ed483a8f2f69fe9e7bfff9"
format_stamp: "Formatted at 2026-08-07 11:28:57 on dist-test-slave-2w3w"
server_key: "fa006871ce665d4b2c96c43775cc13b2"
server_key_iv: "39370c64b4764d394cb7de596a702fb4"
server_key_version: "encryptionkey@0"
I20260807 11:28:57.314643 8187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "e09789d199ed483a8f2f69fe9e7bfff9"
format_stamp: "Formatted at 2026-08-07 11:28:57 on dist-test-slave-2w3w"
server_key: "fa006871ce665d4b2c96c43775cc13b2"
server_key_iv: "39370c64b4764d394cb7de596a702fb4"
server_key_version: "encryptionkey@0"
I20260807 11:28:57.320246 8187 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 11:28:57.323828 8203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:57.325158 8187 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 11:28:57.325341 8187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "e09789d199ed483a8f2f69fe9e7bfff9"
format_stamp: "Formatted at 2026-08-07 11:28:57 on dist-test-slave-2w3w"
server_key: "fa006871ce665d4b2c96c43775cc13b2"
server_key_iv: "39370c64b4764d394cb7de596a702fb4"
server_key_version: "encryptionkey@0"
I20260807 11:28:57.325559 8187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:57.339761 8187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:57.344545 8187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:57.344887 8187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:57.364511 8187 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:33273
I20260807 11:28:57.364534 8258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:33273 every 8 connection(s)
I20260807 11:28:57.365973 8187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:28:57.368000 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 8187
I20260807 11:28:57.368198 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:28:57.368633 17172 external_mini_cluster.cc:1428] Setting key d02a425be44c776106bcee1d5fe63998
I20260807 11:28:57.370609 8259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:57.382671 8262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:57.370456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34763 (local address 127.16.197.62:33273)
0807 11:28:57.371317 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:57.371329 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:57.371372 (+ 43us) server_negotiation.cc:407] Connection header received
0807 11:28:57.373011 (+ 1639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:57.373031 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:57.373436 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:57.373770 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:57.374417 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:57.375862 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:57.377753 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:57.378437 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:57.378570 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:57.379962 (+ 1392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:57.379989 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:57.380004 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:57.380610 (+ 606us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:57.380687 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:57.380721 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:57.380817 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:57.381405 (+ 588us) server_negotiation.cc:299] Negotiation successful
0807 11:28:57.381664 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":417,"threads_started":1}
I20260807 11:28:57.384074 8259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e09789d199ed483a8f2f69fe9e7bfff9: Bootstrap starting.
I20260807 11:28:57.386600 17172 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 11:28:57.387665 8259 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e09789d199ed483a8f2f69fe9e7bfff9: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:57.388695 8259 log.cc:824] T 00000000000000000000000000000000 P e09789d199ed483a8f2f69fe9e7bfff9: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:57.391840 8259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e09789d199ed483a8f2f69fe9e7bfff9: No bootstrap required, opened a new log
I20260807 11:28:57.398298 8262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:57.391217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57047 (local address 127.16.197.62:33273)
0807 11:28:57.391415 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:57.391424 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:57.392019 (+ 595us) server_negotiation.cc:407] Connection header received
0807 11:28:57.392229 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:57.392235 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:57.392328 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:57.392473 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:57.392999 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:57.394100 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:57.395537 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:28:57.396272 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:28:57.396407 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:28:57.397486 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:57.397493 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:57.397498 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:57.397521 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:57.397574 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:57.397595 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:57.397701 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:57.397805 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 11:28:57.397912 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 11:28:57.401567 8259 raft_consensus.cc:348] T 00000000000000000000000000000000 P e09789d199ed483a8f2f69fe9e7bfff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09789d199ed483a8f2f69fe9e7bfff9" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33273 } }
I20260807 11:28:57.401938 8259 raft_consensus.cc:374] T 00000000000000000000000000000000 P e09789d199ed483a8f2f69fe9e7bfff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:57.402012 8259 raft_consensus.cc:729] T 00000000000000000000000000000000 P e09789d199ed483a8f2f69fe9e7bfff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e09789d199ed483a8f2f69fe9e7bfff9, State: Initialized, Role: FOLLOWER
I20260807 11:28:57.402580 8259 consensus_queue.cc:260] T 00000000000000000000000000000000 P e09789d199ed483a8f2f69fe9e7bfff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09789d199ed483a8f2f69fe9e7bfff9" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33273 } }
I20260807 11:28:57.402791 8259 raft_consensus.cc:388] T 00000000000000000000000000000000 P e09789d199ed483a8f2f69fe9e7bfff9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:28:57.402896 8259 raft_consensus.cc:482] T 00000000000000000000000000000000 P e09789d199ed483a8f2f69fe9e7bfff9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:28:57.403095 8259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e09789d199ed483a8f2f69fe9e7bfff9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:57.405517 8259 raft_consensus.cc:504] T 00000000000000000000000000000000 P e09789d199ed483a8f2f69fe9e7bfff9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09789d199ed483a8f2f69fe9e7bfff9" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33273 } }
I20260807 11:28:57.406001 8259 leader_election.cc:302] T 00000000000000000000000000000000 P e09789d199ed483a8f2f69fe9e7bfff9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e09789d199ed483a8f2f69fe9e7bfff9; no voters:
I20260807 11:28:57.406615 8259 leader_election.cc:288] T 00000000000000000000000000000000 P e09789d199ed483a8f2f69fe9e7bfff9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:28:57.406769 8269 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e09789d199ed483a8f2f69fe9e7bfff9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:57.408872 8269 raft_consensus.cc:686] T 00000000000000000000000000000000 P e09789d199ed483a8f2f69fe9e7bfff9 [term 1 LEADER]: Becoming Leader. State: Replica: e09789d199ed483a8f2f69fe9e7bfff9, State: Running, Role: LEADER
I20260807 11:28:57.409379 8259 sys_catalog.cc:564] T 00000000000000000000000000000000 P e09789d199ed483a8f2f69fe9e7bfff9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:28:57.409515 8269 consensus_queue.cc:237] T 00000000000000000000000000000000 P e09789d199ed483a8f2f69fe9e7bfff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09789d199ed483a8f2f69fe9e7bfff9" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33273 } }
I20260807 11:28:57.413901 8270 sys_catalog.cc:455] T 00000000000000000000000000000000 P e09789d199ed483a8f2f69fe9e7bfff9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e09789d199ed483a8f2f69fe9e7bfff9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09789d199ed483a8f2f69fe9e7bfff9" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33273 } } }
I20260807 11:28:57.414287 8270 sys_catalog.cc:458] T 00000000000000000000000000000000 P e09789d199ed483a8f2f69fe9e7bfff9 [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:57.414841 8271 sys_catalog.cc:455] T 00000000000000000000000000000000 P e09789d199ed483a8f2f69fe9e7bfff9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e09789d199ed483a8f2f69fe9e7bfff9. Latest consensus state: current_term: 1 leader_uuid: "e09789d199ed483a8f2f69fe9e7bfff9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09789d199ed483a8f2f69fe9e7bfff9" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33273 } } }
I20260807 11:28:57.415206 8271 sys_catalog.cc:458] T 00000000000000000000000000000000 P e09789d199ed483a8f2f69fe9e7bfff9 [sys.catalog]: This master's current role is: LEADER
W20260807 11:28:57.421180 8285 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e09789d199ed483a8f2f69fe9e7bfff9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 11:28:57.421301 8285 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 11:28:57.421474 8279 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:28:57.423707 8279 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:28:57.433105 8279 catalog_manager.cc:1347] Generated new cluster ID: c87a423f14734929981b0e6202af0e0e
I20260807 11:28:57.433244 8279 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:28:57.444938 8279 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:28:57.446337 8279 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:28:57.456718 8279 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e09789d199ed483a8f2f69fe9e7bfff9: Generated new TSK 0
I20260807 11:28:57.457656 8279 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:28:57.464736 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 8187
W20260807 11:28:57.466041 8254 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34763
W20260807 11:28:57.466864 17172 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 8187: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 11:28:57.467046 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 8187
2026-08-07T11:28:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2043 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:28:57 dist-test-slave-2w3w krb5kdc[8290](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:28:57 dist-test-slave-2w3w krb5kdc[8290](info): set up 2 sockets
Aug 07 11:28:57 dist-test-slave-2w3w krb5kdc[8290](info): commencing operation
krb5kdc: starting...
W20260807 11:28:58.629845 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.003s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:28:58 dist-test-slave-2w3w krb5kdc[8290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:28:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:28:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:28:58.938935 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:36247
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:46131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:36247
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:28:59.296909 8304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:28:59.297233 8304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:28:59.297333 8304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:28:59.304699 8304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:28:59.304822 8304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:28:59.304869 8304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:28:59.304963 8304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:28:59.305038 8304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:28:59.315183 8304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:46131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:36247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:36247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:28:59.316479 8304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:28:59.317844 8304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:28:59.329800 8309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:59.329761 8310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:28:59.329862 8312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:28:59.330557 8304 server_base.cc:1034] running on GCE node
I20260807 11:28:59.331414 8304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:28:59.332875 8304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:28:59.334111 8304 hybrid_clock.cc:648] HybridClock initialized: now 1786102139334062 us; error 47 us; skew 500 ppm
I20260807 11:28:59.334467 8304 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:28:59 dist-test-slave-2w3w krb5kdc[8290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102139, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:28:59.337697 8304 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:28:59.339347 8304 webserver.cc:460] Webserver started at http://127.16.197.62:36813/ using document root <none> and password file <none>
I20260807 11:28:59.340057 8304 fs_manager.cc:359] Metadata directory not provided
I20260807 11:28:59.340207 8304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:28:59.340535 8304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:28:59.343662 8304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "903e3af606c84c3ca82537a2b1596c4a"
format_stamp: "Formatted at 2026-08-07 11:28:59 on dist-test-slave-2w3w"
server_key: "eb135b800417692f610e7e7d68353e2d"
server_key_iv: "3d0a0d47d4bc7d7cbb8e5e68b3fe21d7"
server_key_version: "encryptionkey@0"
I20260807 11:28:59.344463 8304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "903e3af606c84c3ca82537a2b1596c4a"
format_stamp: "Formatted at 2026-08-07 11:28:59 on dist-test-slave-2w3w"
server_key: "eb135b800417692f610e7e7d68353e2d"
server_key_iv: "3d0a0d47d4bc7d7cbb8e5e68b3fe21d7"
server_key_version: "encryptionkey@0"
I20260807 11:28:59.349856 8304 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 11:28:59.353298 8320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:28:59.354646 8304 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 11:28:59.354858 8304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "903e3af606c84c3ca82537a2b1596c4a"
format_stamp: "Formatted at 2026-08-07 11:28:59 on dist-test-slave-2w3w"
server_key: "eb135b800417692f610e7e7d68353e2d"
server_key_iv: "3d0a0d47d4bc7d7cbb8e5e68b3fe21d7"
server_key_version: "encryptionkey@0"
I20260807 11:28:59.355122 8304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:28:59.361194 8304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:28:59.365397 8304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:28:59.365731 8304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:28:59.384589 8304 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:36247
I20260807 11:28:59.384624 8374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:36247 every 8 connection(s)
I20260807 11:28:59.386075 8304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:28:59.389372 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 8304
I20260807 11:28:59.389569 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:28:59.390153 17172 external_mini_cluster.cc:1428] Setting key c13971aa2e3d43054b245457421f1407
I20260807 11:28:59.390388 8375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:28:59.398581 8378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:59.392095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34975 (local address 127.16.197.62:36247)
0807 11:28:59.392885 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:59.392896 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:59.392946 (+ 50us) server_negotiation.cc:407] Connection header received
0807 11:28:59.395180 (+ 2234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:59.395204 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:59.395527 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:59.395906 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:59.397003 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:59.397012 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:59.397024 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:59.397038 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:59.397094 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:59.397115 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:59.397368 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:59.397515 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 11:28:59.397693 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":343,"threads_started":1}
I20260807 11:28:59.402227 17172 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 11:28:59.403743 8375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 903e3af606c84c3ca82537a2b1596c4a: Bootstrap starting.
I20260807 11:28:59.407483 8375 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 903e3af606c84c3ca82537a2b1596c4a: Neither blocks nor log segments found. Creating new log.
I20260807 11:28:59.408716 8375 log.cc:824] T 00000000000000000000000000000000 P 903e3af606c84c3ca82537a2b1596c4a: Log is configured to *not* fsync() on all Append() calls
I20260807 11:28:59.409142 8378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:28:59.406575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57529 (local address 127.16.197.62:36247)
0807 11:28:59.406774 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 11:28:59.406782 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:28:59.407197 (+ 415us) server_negotiation.cc:407] Connection header received
0807 11:28:59.407401 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:28:59.407408 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:28:59.407471 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:28:59.407574 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:28:59.408516 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:28:59.408521 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:28:59.408528 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 11:28:59.408548 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:28:59.408588 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 11:28:59.408603 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:28:59.408725 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 11:28:59.408817 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 11:28:59.408876 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 11:28:59.411950 8375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 903e3af606c84c3ca82537a2b1596c4a: No bootstrap required, opened a new log
I20260807 11:28:59.419919 8375 raft_consensus.cc:348] T 00000000000000000000000000000000 P 903e3af606c84c3ca82537a2b1596c4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "903e3af606c84c3ca82537a2b1596c4a" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 36247 } }
I20260807 11:28:59.420456 8375 raft_consensus.cc:374] T 00000000000000000000000000000000 P 903e3af606c84c3ca82537a2b1596c4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 11:28:59.420576 8375 raft_consensus.cc:729] T 00000000000000000000000000000000 P 903e3af606c84c3ca82537a2b1596c4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903e3af606c84c3ca82537a2b1596c4a, State: Initialized, Role: FOLLOWER
I20260807 11:28:59.421197 8375 consensus_queue.cc:260] T 00000000000000000000000000000000 P 903e3af606c84c3ca82537a2b1596c4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "903e3af606c84c3ca82537a2b1596c4a" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 36247 } }
I20260807 11:28:59.421452 8375 raft_consensus.cc:388] T 00000000000000000000000000000000 P 903e3af606c84c3ca82537a2b1596c4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 11:28:59.421610 8375 raft_consensus.cc:482] T 00000000000000000000000000000000 P 903e3af606c84c3ca82537a2b1596c4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 11:28:59.421795 8375 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 903e3af606c84c3ca82537a2b1596c4a [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:28:59.424434 8375 raft_consensus.cc:504] T 00000000000000000000000000000000 P 903e3af606c84c3ca82537a2b1596c4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "903e3af606c84c3ca82537a2b1596c4a" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 36247 } }
I20260807 11:28:59.424960 8375 leader_election.cc:302] T 00000000000000000000000000000000 P 903e3af606c84c3ca82537a2b1596c4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 903e3af606c84c3ca82537a2b1596c4a; no voters:
I20260807 11:28:59.425591 8375 leader_election.cc:288] T 00000000000000000000000000000000 P 903e3af606c84c3ca82537a2b1596c4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 11:28:59.425727 8385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 903e3af606c84c3ca82537a2b1596c4a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:28:59.426410 8385 raft_consensus.cc:686] T 00000000000000000000000000000000 P 903e3af606c84c3ca82537a2b1596c4a [term 1 LEADER]: Becoming Leader. State: Replica: 903e3af606c84c3ca82537a2b1596c4a, State: Running, Role: LEADER
I20260807 11:28:59.427029 8385 consensus_queue.cc:237] T 00000000000000000000000000000000 P 903e3af606c84c3ca82537a2b1596c4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "903e3af606c84c3ca82537a2b1596c4a" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 36247 } }
I20260807 11:28:59.427955 8375 sys_catalog.cc:564] T 00000000000000000000000000000000 P 903e3af606c84c3ca82537a2b1596c4a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:28:59.430138 8387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 903e3af606c84c3ca82537a2b1596c4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 903e3af606c84c3ca82537a2b1596c4a. Latest consensus state: current_term: 1 leader_uuid: "903e3af606c84c3ca82537a2b1596c4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "903e3af606c84c3ca82537a2b1596c4a" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 36247 } } }
I20260807 11:28:59.430613 8387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 903e3af606c84c3ca82537a2b1596c4a [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:59.431231 8386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 903e3af606c84c3ca82537a2b1596c4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "903e3af606c84c3ca82537a2b1596c4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "903e3af606c84c3ca82537a2b1596c4a" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 36247 } } }
I20260807 11:28:59.431677 8386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 903e3af606c84c3ca82537a2b1596c4a [sys.catalog]: This master's current role is: LEADER
I20260807 11:28:59.434156 8394 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:28:59.439579 8394 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:28:59.448386 8394 catalog_manager.cc:1347] Generated new cluster ID: cd7c4e943bd9445f9c88ed2fcc25bb58
I20260807 11:28:59.448524 8394 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:28:59.485551 8394 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:28:59.486080 8394 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:28:59.498569 8394 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 903e3af606c84c3ca82537a2b1596c4a: Generated new TSK 0
I20260807 11:28:59.499614 8394 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:28:59.513199 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 8304
W20260807 11:28:59.514423 8370 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34975
W20260807 11:28:59.515177 17172 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 8304: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 11:28:59.515336 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 8304
2026-08-07T11:28:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2047 ms)
[----------] 18 tests from AuthTokenIssuingTest (37002 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:28:59 dist-test-slave-2w3w krb5kdc[8407](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:28:59 dist-test-slave-2w3w krb5kdc[8407](info): set up 2 sockets
Aug 07 11:28:59 dist-test-slave-2w3w krb5kdc[8407](info): commencing operation
krb5kdc: starting...
W20260807 11:29:00.673700 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:29:00 dist-test-slave-2w3w krb5kdc[8407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:29:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:29:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:29:00.974288 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:33131
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:43219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:33131,127.16.197.61:34783,127.16.197.60:35135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:29:01.343127 8420 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 11:29:01.343463 8420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:29:01.343538 8420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:29:01.343621 8420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:29:01.351094 8420 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:29:01.351210 8420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:29:01.351277 8420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:29:01.351331 8420 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:29:01.351387 8420 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:29:01.361564 8420 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:43219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:33131,127.16.197.61:34783,127.16.197.60:35135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:33131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:29:01.362852 8420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:29:01.364337 8420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:29:01.376888 8425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:29:01.376813 8426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:29:01.376845 8428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:29:01.377734 8420 server_base.cc:1034] running on GCE node
I20260807 11:29:01.378551 8420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:29:01.380028 8420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:29:01.381284 8420 hybrid_clock.cc:648] HybridClock initialized: now 1786102141381232 us; error 27 us; skew 500 ppm
I20260807 11:29:01.381623 8420 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:29:01 dist-test-slave-2w3w krb5kdc[8407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102141, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:29:01.384878 8420 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:29:01.386371 8420 webserver.cc:460] Webserver started at http://127.16.197.62:37017/ using document root <none> and password file <none>
I20260807 11:29:01.387107 8420 fs_manager.cc:359] Metadata directory not provided
I20260807 11:29:01.387199 8420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:29:01.387467 8420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:29:01.390372 8420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "2ad7ae4b264f45589313e987fadb8b99"
format_stamp: "Formatted at 2026-08-07 11:29:01 on dist-test-slave-2w3w"
server_key: "9940f3e0af8a143be1cfaf27eca30e78"
server_key_iv: "c11ba51a9773f48a741df063fc0f1c14"
server_key_version: "encryptionkey@0"
I20260807 11:29:01.391172 8420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "2ad7ae4b264f45589313e987fadb8b99"
format_stamp: "Formatted at 2026-08-07 11:29:01 on dist-test-slave-2w3w"
server_key: "9940f3e0af8a143be1cfaf27eca30e78"
server_key_iv: "c11ba51a9773f48a741df063fc0f1c14"
server_key_version: "encryptionkey@0"
I20260807 11:29:01.396245 8420 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 11:29:01.399616 8435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:29:01.400951 8420 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 11:29:01.401093 8420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "2ad7ae4b264f45589313e987fadb8b99"
format_stamp: "Formatted at 2026-08-07 11:29:01 on dist-test-slave-2w3w"
server_key: "9940f3e0af8a143be1cfaf27eca30e78"
server_key_iv: "c11ba51a9773f48a741df063fc0f1c14"
server_key_version: "encryptionkey@0"
I20260807 11:29:01.401280 8420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:29:01.439059 8420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:29:01.443142 8420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:29:01.443509 8420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:29:01.462702 8420 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:33131
I20260807 11:29:01.462723 8489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:33131 every 8 connection(s)
I20260807 11:29:01.464169 8420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:29:01.466276 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 8420
I20260807 11:29:01.466503 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:29:01.467211 17172 external_mini_cluster.cc:1428] Setting key b36ad9ca85a03e11cbe5850dc6892452
I20260807 11:29:01.468252 8490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:29:01.475253 8490 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33131 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 11:29:01 dist-test-slave-2w3w krb5kdc[8407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102141, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.197.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.61@KRBTEST.COM" created.
I20260807 11:29:01.501315 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:01.478628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33131 (local address 127.0.0.1:60934)
0807 11:29:01.479604 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:01.479666 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 11:29:01.482587 (+ 2921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:29:01.483620 (+ 1033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:29:01.483630 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:29:01.484553 (+ 923us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:29:01.485088 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:01.485117 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:01.486713 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:01.486721 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:01.488758 (+ 2037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:01.488777 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:01.489762 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:01.489772 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:01.490008 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:01.490748 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:29:01.490798 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:29:01.493628 (+ 2830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:29:01.496939 (+ 3311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:01.496951 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:01.496964 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:01.497433 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:01.497850 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:01.497854 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:01.497858 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:01.498020 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:01.498785 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:29:01.498800 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:29:01.499085 (+ 285us) client_negotiation.cc:769] Sending connection context
0807 11:29:01.499334 (+ 249us) client_negotiation.cc:240] Negotiation successful
0807 11:29:01.499958 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"mutex_wait_us":144,"thread_start_us":403,"threads_started":1}
I20260807 11:29:01.501353 8495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:01.478628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60934 (local address 127.16.197.62:33131)
0807 11:29:01.480622 (+ 1994us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:01.480627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:01.480675 (+ 48us) server_negotiation.cc:407] Connection header received
0807 11:29:01.483082 (+ 2407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:01.483088 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:01.483334 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:01.483519 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:01.485323 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:01.486536 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:01.488918 (+ 2382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:01.489492 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:01.489624 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:01.493859 (+ 4235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:01.493867 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:01.493879 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:01.493912 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:01.496780 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:01.497576 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:01.497582 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:01.497584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:01.497728 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:01.498136 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:01.498140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:01.498145 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:01.498649 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:01.498871 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:01.499480 (+ 609us) server_negotiation.cc:299] Negotiation successful
0807 11:29:01.499798 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1807,"spinlock_wait_cycles":61824,"thread_start_us":1524,"threads_started":1}
I20260807 11:29:01.504501 8490 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.61" port: 34783 } has no permanent_uuid. Determining permanent_uuid...
I20260807 11:29:01.505559 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:01.505150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:34783 (local address 127.0.0.1:60288)
0807 11:29:01.505343 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:01.505446 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:34783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 11:29:01.505788 8438 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.197.61:34783: connect: Connection refused (error 111)
I20260807 11:29:01.507493 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:01.507118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:34783 (local address 127.0.0.1:60294)
0807 11:29:01.507304 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:01.507368 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:34783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 11:29:01.507691 8490 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.61:34783: Network error: Client connection negotiation failed: client connection to 127.16.197.61:34783: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.61.keytab.
Entry for principal kudu/127.16.197.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.61.keytab.
I20260807 11:29:01.535054 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.61:34783
--webserver_interface=127.16.197.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.61.keytab
--principal=kudu/127.16.197.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:43219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:33131,127.16.197.61:34783,127.16.197.60:35135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 11:29:01.557068 8490 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.61" port: 34783 } attempt: 1
I20260807 11:29:01.558543 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:01.557910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:34783 (local address 127.0.0.1:60310)
0807 11:29:01.558180 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:01.558331 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:34783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 11:29:01.559764 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:01.559292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:34783 (local address 127.0.0.1:60326)
0807 11:29:01.559472 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:01.559572 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:34783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 11:29:01.559976 8490 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.61:34783: Network error: Client connection negotiation failed: client connection to 127.16.197.61:34783: connect: Connection refused (error 111)
I20260807 11:29:01.628297 8490 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.61" port: 34783 } attempt: 2
I20260807 11:29:01.629818 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:01.629128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:34783 (local address 127.0.0.1:60330)
0807 11:29:01.629488 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:01.629622 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:34783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260807 11:29:01.631017 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:01.630549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:34783 (local address 127.0.0.1:60332)
0807 11:29:01.630707 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:01.630814 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:34783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 11:29:01.631235 8490 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.61:34783: Network error: Client connection negotiation failed: client connection to 127.16.197.61:34783: connect: Connection refused (error 111)
I20260807 11:29:01.722551 8490 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.61" port: 34783 } attempt: 3
I20260807 11:29:01.723968 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:01.723344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:34783 (local address 127.0.0.1:60342)
0807 11:29:01.723619 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:01.723760 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:34783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 11:29:01.725188 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:01.724742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:34783 (local address 127.0.0.1:60352)
0807 11:29:01.724912 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:01.725004 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:34783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 11:29:01.725389 8490 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.61:34783: Network error: Client connection negotiation failed: client connection to 127.16.197.61:34783: connect: Connection refused (error 111)
I20260807 11:29:01.868672 8490 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.61" port: 34783 } attempt: 4
I20260807 11:29:01.870148 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:01.869444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:34783 (local address 127.0.0.1:60362)
0807 11:29:01.869846 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:01.869965 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:34783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206}
I20260807 11:29:01.871438 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:01.870934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:34783 (local address 127.0.0.1:60374)
0807 11:29:01.871169 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:01.871281 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:34783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
W20260807 11:29:01.871644 8490 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.61:34783: Network error: Client connection negotiation failed: client connection to 127.16.197.61:34783: connect: Connection refused (error 111)
W20260807 11:29:01.883088 8498 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 11:29:01.883447 8498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:29:01.883524 8498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:29:01.883613 8498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:29:01.891187 8498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:29:01.891355 8498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:29:01.891495 8498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:29:01.891595 8498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:29:01.891690 8498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:29:01.902042 8498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:43219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:33131,127.16.197.61:34783,127.16.197.60:35135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.61.keytab
--principal=kudu/127.16.197.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.61:34783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.16.197.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:29:01.903594 8498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:29:01.905077 8498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:29:01.917778 8504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:29:01.917801 8505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:29:01.917829 8507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:29:01.918573 8498 server_base.cc:1034] running on GCE node
I20260807 11:29:01.919486 8498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:29:01.920964 8498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:29:01.922237 8498 hybrid_clock.cc:648] HybridClock initialized: now 1786102141922143 us; error 60 us; skew 500 ppm
I20260807 11:29:01.922663 8498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:29:01 dist-test-slave-2w3w krb5kdc[8407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102141, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:29:01.926007 8498 init.cc:377] Logged in from keytab as kudu/127.16.197.61@KRBTEST.COM (short username kudu)
I20260807 11:29:01.927659 8498 webserver.cc:460] Webserver started at http://127.16.197.61:36459/ using document root <none> and password file <none>
I20260807 11:29:01.928371 8498 fs_manager.cc:359] Metadata directory not provided
I20260807 11:29:01.928508 8498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:29:01.928835 8498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:29:01.931960 8498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-1/data/instance:
uuid: "2f108e02dc194b7c94891ce8b033cb4e"
format_stamp: "Formatted at 2026-08-07 11:29:01 on dist-test-slave-2w3w"
server_key: "955efc0fa2164a8e4cc1ad1143df32d4"
server_key_iv: "46c45a25b34b332fdb02c3ab4fa02182"
server_key_version: "encryptionkey@0"
I20260807 11:29:01.932804 8498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-1/wal/instance:
uuid: "2f108e02dc194b7c94891ce8b033cb4e"
format_stamp: "Formatted at 2026-08-07 11:29:01 on dist-test-slave-2w3w"
server_key: "955efc0fa2164a8e4cc1ad1143df32d4"
server_key_iv: "46c45a25b34b332fdb02c3ab4fa02182"
server_key_version: "encryptionkey@0"
I20260807 11:29:01.938321 8498 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 11:29:01.941890 8515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:29:01.943270 8498 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:29:01.943464 8498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-1/wal
uuid: "2f108e02dc194b7c94891ce8b033cb4e"
format_stamp: "Formatted at 2026-08-07 11:29:01 on dist-test-slave-2w3w"
server_key: "955efc0fa2164a8e4cc1ad1143df32d4"
server_key_iv: "46c45a25b34b332fdb02c3ab4fa02182"
server_key_version: "encryptionkey@0"
I20260807 11:29:01.943708 8498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:29:01.966007 8498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:29:01.970575 8498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:29:01.970888 8498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:29:01.991176 8498 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.61:34783
I20260807 11:29:01.991214 8566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.61:34783 every 8 connection(s)
I20260807 11:29:01.992615 8498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-1/data/info.pb
I20260807 11:29:01.995131 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 8498
I20260807 11:29:01.995342 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-1/wal/instance
I20260807 11:29:01.995915 17172 external_mini_cluster.cc:1428] Setting key bf74d625883c60a466eb873b69f518fe
I20260807 11:29:01.997215 8567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:29:02.007782 8567 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33131 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 11:29:02 dist-test-slave-2w3w krb5kdc[8407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102141, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.61@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.197.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.60@KRBTEST.COM" created.
I20260807 11:29:02.034583 8570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.011447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60948 (local address 127.16.197.62:33131)
0807 11:29:02.012044 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:02.012051 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:02.013088 (+ 1037us) server_negotiation.cc:407] Connection header received
0807 11:29:02.016124 (+ 3036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:02.016130 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:02.016202 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:02.016306 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:02.018563 (+ 2257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.019468 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.022039 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.022467 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.022570 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:02.027217 (+ 4647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:02.027227 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:02.027235 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:02.027293 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:02.031163 (+ 3870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:02.031952 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:02.031957 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:02.031959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:02.032120 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:02.032599 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:02.032602 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:02.032604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:02.032887 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:02.033011 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:02.033868 (+ 857us) server_negotiation.cc:299] Negotiation successful
0807 11:29:02.034110 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":302,"threads_started":1}
I20260807 11:29:02.035902 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.012034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33131 (local address 127.0.0.1:60948)
0807 11:29:02.012886 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:02.012938 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 11:29:02.015810 (+ 2872us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:29:02.016636 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:29:02.016650 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:29:02.017461 (+ 811us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:29:02.018328 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:02.018357 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.019651 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.019659 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:02.021869 (+ 2210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:02.021887 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.022726 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.022735 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:02.023094 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:02.023892 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:29:02.023932 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:29:02.026894 (+ 2962us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:29:02.031335 (+ 4441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:02.031342 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:02.031356 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:02.031813 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:02.032263 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:02.032271 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:02.032278 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:02.032497 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:02.033016 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:29:02.033022 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:29:02.033539 (+ 517us) client_negotiation.cc:769] Sending connection context
0807 11:29:02.033796 (+ 257us) client_negotiation.cc:240] Negotiation successful
0807 11:29:02.034320 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":312,"threads_started":1}
I20260807 11:29:02.037884 8567 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.61" port: 34783 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 11:29:02 dist-test-slave-2w3w krb5kdc[8407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102141, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.61@KRBTEST.COM for kudu/127.16.197.61@KRBTEST.COM
I20260807 11:29:02.055685 8573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.038837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60382 (local address 127.16.197.61:34783)
0807 11:29:02.039624 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:02.039630 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:02.039656 (+ 26us) server_negotiation.cc:407] Connection header received
0807 11:29:02.039698 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:02.039704 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:02.039869 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:02.039986 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:02.040989 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.042032 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.043846 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.044454 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.044637 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:02.048552 (+ 3915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:02.048567 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:02.048586 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:02.048634 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:02.052677 (+ 4043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:02.053409 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:02.053427 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:02.053429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:02.053548 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:02.054036 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:02.054039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:02.054041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:02.054381 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:02.054550 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:02.055055 (+ 505us) server_negotiation.cc:299] Negotiation successful
0807 11:29:02.055361 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":93,"server-negotiator.queue_time_us":636,"thread_start_us":479,"threads_started":1}
I20260807 11:29:02.055766 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.038589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:34783 (local address 127.0.0.1:60382)
0807 11:29:02.038802 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:02.038825 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 11:29:02.038947 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:29:02.040044 (+ 1097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:29:02.040052 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:29:02.040585 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:29:02.040808 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:02.040824 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.042182 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.042190 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:02.043702 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:02.043720 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.044657 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.044666 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:02.044910 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:02.045543 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:29:02.045570 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:29:02.048334 (+ 2764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:29:02.052827 (+ 4493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:02.052831 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:02.052834 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:02.053210 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:02.053664 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:02.053671 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:02.053677 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:02.053913 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:02.054516 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:29:02.054523 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:29:02.054694 (+ 171us) client_negotiation.cc:769] Sending connection context
0807 11:29:02.054864 (+ 170us) client_negotiation.cc:240] Negotiation successful
0807 11:29:02.055244 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 11:29:02.058080 8567 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } has no permanent_uuid. Determining permanent_uuid...
I20260807 11:29:02.059252 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.058761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:35135 (local address 127.0.0.1:35548)
0807 11:29:02.058921 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:02.059090 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:35135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 11:29:02.059581 8518 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.197.60:35135: connect: Connection refused (error 111)
I20260807 11:29:02.061074 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.060770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:35135 (local address 127.0.0.1:35562)
0807 11:29:02.060905 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:02.060980 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:35135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 11:29:02.061373 8567 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.60:35135: Network error: Client connection negotiation failed: client connection to 127.16.197.60:35135: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.60.keytab.
Entry for principal kudu/127.16.197.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.60.keytab.
I20260807 11:29:02.075080 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.60:35135
--webserver_interface=127.16.197.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.60.keytab
--principal=kudu/127.16.197.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:43219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:33131,127.16.197.61:34783,127.16.197.60:35135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 11:29:02.110721 8567 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } attempt: 1
I20260807 11:29:02.111938 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.111465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:35135 (local address 127.0.0.1:35570)
0807 11:29:02.111664 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:02.111782 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:35135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 11:29:02.113003 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.112651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:35135 (local address 127.0.0.1:35582)
0807 11:29:02.112826 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:02.112897 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:35135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 11:29:02.113220 8567 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.60:35135: Network error: Client connection negotiation failed: client connection to 127.16.197.60:35135: connect: Connection refused (error 111)
I20260807 11:29:02.171057 8490 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.61" port: 34783 } attempt: 5
Aug 07 11:29:02 dist-test-slave-2w3w krb5kdc[8407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102141, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for kudu/127.16.197.61@KRBTEST.COM
I20260807 11:29:02.181469 8567 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } attempt: 2
I20260807 11:29:02.182639 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.182119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:35135 (local address 127.0.0.1:35586)
0807 11:29:02.182342 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:02.182455 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:35135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 11:29:02.183760 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.183349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:35135 (local address 127.0.0.1:35596)
0807 11:29:02.183522 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:02.183612 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:35135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 11:29:02.184026 8567 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.60:35135: Network error: Client connection negotiation failed: client connection to 127.16.197.60:35135: connect: Connection refused (error 111)
I20260807 11:29:02.185333 8573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.172065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60386 (local address 127.16.197.61:34783)
0807 11:29:02.172252 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:02.172260 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:02.172282 (+ 22us) server_negotiation.cc:407] Connection header received
0807 11:29:02.172493 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:02.172500 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:02.172583 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:02.172695 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:02.173624 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.174335 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.175616 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.176048 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.176183 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:02.179366 (+ 3183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:02.179371 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:02.179374 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:02.179407 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:02.182385 (+ 2978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:02.183258 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:02.183263 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:02.183267 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:02.183415 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:02.183832 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:02.183837 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:02.183842 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:02.184257 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:02.184414 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:02.184816 (+ 402us) server_negotiation.cc:299] Negotiation successful
0807 11:29:02.185034 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 11:29:02.185379 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.171950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:34783 (local address 127.0.0.1:60386)
0807 11:29:02.172182 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:02.172204 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 11:29:02.172337 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:29:02.172743 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:29:02.172748 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:29:02.173269 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:29:02.173464 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:02.173474 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.174497 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.174503 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:02.175474 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:02.175486 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.176201 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.176207 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:02.176441 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:02.177093 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:29:02.177112 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:29:02.179172 (+ 2060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:29:02.182538 (+ 3366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:02.182544 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:02.182548 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:02.183143 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:02.183564 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:02.183568 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:02.183570 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:02.183707 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:02.184380 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:29:02.184385 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:29:02.184551 (+ 166us) client_negotiation.cc:769] Sending connection context
0807 11:29:02.184690 (+ 139us) client_negotiation.cc:240] Negotiation successful
0807 11:29:02.185029 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 11:29:02.186720 8490 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } has no permanent_uuid. Determining permanent_uuid...
I20260807 11:29:02.187793 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.187379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:35135 (local address 127.0.0.1:35612)
0807 11:29:02.187559 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:02.187649 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:35135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 11:29:02.188699 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.188366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:35135 (local address 127.0.0.1:35616)
0807 11:29:02.188517 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:02.188595 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:35135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 11:29:02.188931 8490 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.60:35135: Network error: Client connection negotiation failed: client connection to 127.16.197.60:35135: connect: Connection refused (error 111)
I20260807 11:29:02.240149 8490 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } attempt: 1
I20260807 11:29:02.241405 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.240949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:35135 (local address 127.0.0.1:35630)
0807 11:29:02.241148 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:02.241241 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:35135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 11:29:02.242424 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.242086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:35135 (local address 127.0.0.1:35646)
0807 11:29:02.242247 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:02.242316 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:35135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 11:29:02.242637 8490 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.60:35135: Network error: Client connection negotiation failed: client connection to 127.16.197.60:35135: connect: Connection refused (error 111)
I20260807 11:29:02.275285 8567 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } attempt: 3
I20260807 11:29:02.276719 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.276019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:35135 (local address 127.0.0.1:35654)
0807 11:29:02.276337 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:02.276475 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:35135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260807 11:29:02.278110 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.277516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:35135 (local address 127.0.0.1:35664)
0807 11:29:02.277800 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:02.277934 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:35135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
W20260807 11:29:02.278402 8567 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.60:35135: Network error: Client connection negotiation failed: client connection to 127.16.197.60:35135: connect: Connection refused (error 111)
I20260807 11:29:02.310871 8490 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } attempt: 2
I20260807 11:29:02.312109 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.311617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:35135 (local address 127.0.0.1:35670)
0807 11:29:02.311845 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:02.311944 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:35135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 11:29:02.313060 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.312737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:35135 (local address 127.0.0.1:35686)
0807 11:29:02.312879 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:02.312960 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:35135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 11:29:02.313264 8490 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.60:35135: Network error: Client connection negotiation failed: client connection to 127.16.197.60:35135: connect: Connection refused (error 111)
I20260807 11:29:02.419483 8490 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } attempt: 3
W20260807 11:29:02.420063 8576 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 11:29:02.420435 8576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:29:02.420565 8576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
I20260807 11:29:02.420660 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.420137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:35135 (local address 127.0.0.1:35702)
0807 11:29:02.420345 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:02.420439 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:35135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260807 11:29:02.420703 8576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 11:29:02.421716 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.421384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:35135 (local address 127.0.0.1:35710)
0807 11:29:02.421552 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:02.421624 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:35135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 11:29:02.421638 8567 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } attempt: 4
W20260807 11:29:02.421963 8490 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.60:35135: Network error: Client connection negotiation failed: client connection to 127.16.197.60:35135: connect: Connection refused (error 111)
I20260807 11:29:02.422552 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.422142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:35135 (local address 127.0.0.1:35724)
0807 11:29:02.422335 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:02.422411 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:35135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 11:29:02.423559 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.423240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:35135 (local address 127.0.0.1:35736)
0807 11:29:02.423387 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:02.423463 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:35135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 11:29:02.423786 8567 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.60:35135: Network error: Client connection negotiation failed: client connection to 127.16.197.60:35135: connect: Connection refused (error 111)
W20260807 11:29:02.428915 8576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:29:02.429018 8576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:29:02.429144 8576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:29:02.429220 8576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:29:02.429313 8576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:29:02.439512 8576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:43219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:33131,127.16.197.61:34783,127.16.197.60:35135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.60.keytab
--principal=kudu/127.16.197.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.60:35135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.16.197.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:29:02.440840 8576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:29:02.442287 8576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:29:02.454265 8581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:29:02.454376 8582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:29:02.454336 8584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:29:02.455257 8576 server_base.cc:1034] running on GCE node
I20260807 11:29:02.455999 8576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:29:02.457314 8576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:29:02.458565 8576 hybrid_clock.cc:648] HybridClock initialized: now 1786102142458510 us; error 52 us; skew 500 ppm
I20260807 11:29:02.458892 8576 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:29:02 dist-test-slave-2w3w krb5kdc[8407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102142, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:29:02.461926 8576 init.cc:377] Logged in from keytab as kudu/127.16.197.60@KRBTEST.COM (short username kudu)
I20260807 11:29:02.463512 8576 webserver.cc:460] Webserver started at http://127.16.197.60:41339/ using document root <none> and password file <none>
I20260807 11:29:02.464182 8576 fs_manager.cc:359] Metadata directory not provided
I20260807 11:29:02.464316 8576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:29:02.464636 8576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:29:02.467576 8576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-2/data/instance:
uuid: "33df89f012a6459d9926e22163ad6af2"
format_stamp: "Formatted at 2026-08-07 11:29:02 on dist-test-slave-2w3w"
server_key: "020d906d7995e0b004fc5df87fc32052"
server_key_iv: "49fb424a2738dc9c628cee8031be7680"
server_key_version: "encryptionkey@0"
I20260807 11:29:02.468369 8576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-2/wal/instance:
uuid: "33df89f012a6459d9926e22163ad6af2"
format_stamp: "Formatted at 2026-08-07 11:29:02 on dist-test-slave-2w3w"
server_key: "020d906d7995e0b004fc5df87fc32052"
server_key_iv: "49fb424a2738dc9c628cee8031be7680"
server_key_version: "encryptionkey@0"
I20260807 11:29:02.473471 8576 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 11:29:02.476995 8591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:29:02.478238 8576 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 11:29:02.478428 8576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-2/wal
uuid: "33df89f012a6459d9926e22163ad6af2"
format_stamp: "Formatted at 2026-08-07 11:29:02 on dist-test-slave-2w3w"
server_key: "020d906d7995e0b004fc5df87fc32052"
server_key_iv: "49fb424a2738dc9c628cee8031be7680"
server_key_version: "encryptionkey@0"
I20260807 11:29:02.478655 8576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:29:02.500068 8576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:29:02.504261 8576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:29:02.504580 8576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:29:02.524096 8576 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.60:35135
I20260807 11:29:02.524127 8645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.60:35135 every 8 connection(s)
I20260807 11:29:02.525657 8576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-2/data/info.pb
I20260807 11:29:02.529323 8646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:29:02.535358 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 8576
I20260807 11:29:02.535548 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786101765888975-17172-0/minicluster-data/master-2/wal/instance
I20260807 11:29:02.536000 17172 external_mini_cluster.cc:1428] Setting key 2827ba4753bfca9a2ed677d255e90a78
I20260807 11:29:02.535972 8646 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33131 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 11:29:02 dist-test-slave-2w3w krb5kdc[8407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102142, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.60@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
Aug 07 11:29:02 dist-test-slave-2w3w krb5kdc[8407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.60@KRBTEST.COM
I20260807 11:29:02.558629 8650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.538741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60952 (local address 127.16.197.62:33131)
0807 11:29:02.539361 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:02.539369 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:02.539936 (+ 567us) server_negotiation.cc:407] Connection header received
0807 11:29:02.541158 (+ 1222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:02.541164 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:02.541245 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:02.541356 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:02.543240 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.544454 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.546602 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.547309 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.547440 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:02.551769 (+ 4329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:02.551777 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:02.551783 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:02.551829 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:02.555164 (+ 3335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:02.556199 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:02.556203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:02.556205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:02.556338 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:02.556807 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:02.556811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:02.556813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:02.557087 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:02.557247 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:02.557954 (+ 707us) server_negotiation.cc:299] Negotiation successful
0807 11:29:02.558183 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":308,"threads_started":1}
I20260807 11:29:02.559878 8649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.537977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35740 (local address 127.16.197.60:35135)
0807 11:29:02.538842 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:02.538856 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:02.538894 (+ 38us) server_negotiation.cc:407] Connection header received
0807 11:29:02.540879 (+ 1985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:02.540900 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:02.541180 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:02.541407 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:02.543313 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.544951 (+ 1638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.546728 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.547643 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.547824 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:02.552773 (+ 4949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:02.552783 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:02.552796 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:02.552830 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:02.556301 (+ 3471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:02.557007 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:02.557016 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:02.557020 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:02.557156 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:02.557573 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:02.557578 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:02.557582 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:02.557976 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:02.558313 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:02.558637 (+ 324us) server_negotiation.cc:299] Negotiation successful
0807 11:29:02.559203 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":87,"server-negotiator.queue_time_us":652,"thread_start_us":388,"threads_started":1}
I20260807 11:29:02.559918 8651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.538762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33131 (local address 127.0.0.1:60952)
0807 11:29:02.539778 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:02.539821 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 11:29:02.540891 (+ 1070us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:29:02.541442 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:29:02.541449 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:29:02.542369 (+ 920us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:29:02.543008 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:02.543041 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.544631 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.544635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:02.546450 (+ 1815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:02.546466 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.547571 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.547576 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:02.547834 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:02.548712 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:29:02.548750 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:29:02.551511 (+ 2761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:29:02.555306 (+ 3795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:02.555317 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:02.555331 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:02.556061 (+ 730us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:02.556465 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:02.556471 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:02.556475 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:02.556673 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:02.557221 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:29:02.557229 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:29:02.557581 (+ 352us) client_negotiation.cc:769] Sending connection context
0807 11:29:02.557860 (+ 279us) client_negotiation.cc:240] Negotiation successful
0807 11:29:02.558389 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":763,"mutex_wait_us":9,"thread_start_us":592,"threads_started":1}
I20260807 11:29:02.561569 8646 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.61" port: 34783 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 11:29:02 dist-test-slave-2w3w krb5kdc[8407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102142, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.60@KRBTEST.COM for kudu/127.16.197.61@KRBTEST.COM
Aug 07 11:29:02 dist-test-slave-2w3w krb5kdc[8407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:29:02.577283 8651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.562270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:34783 (local address 127.0.0.1:60398)
0807 11:29:02.562438 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:02.562453 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 11:29:02.562531 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:29:02.563245 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:29:02.563248 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:29:02.563570 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:29:02.563706 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:02.563714 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.565993 (+ 2279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.565996 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:02.566857 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:02.566868 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.567716 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.567722 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:02.567959 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:02.568666 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:29:02.568683 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:29:02.570882 (+ 2199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:29:02.574775 (+ 3893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:02.574779 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:02.574782 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:02.575243 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:02.575670 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:02.575678 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:02.575682 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:02.575843 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:02.576429 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:29:02.576432 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:29:02.576604 (+ 172us) client_negotiation.cc:769] Sending connection context
0807 11:29:02.576760 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 11:29:02.577027 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 11:29:02.577701 8573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.562539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60398 (local address 127.16.197.61:34783)
0807 11:29:02.562784 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:02.562791 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:02.562810 (+ 19us) server_negotiation.cc:407] Connection header received
0807 11:29:02.562870 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:02.562875 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:02.562954 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:02.563078 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:02.563862 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.564957 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.567135 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.567558 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.567694 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:02.571177 (+ 3483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:02.571186 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:02.571193 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:02.571241 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:02.574618 (+ 3377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:02.575366 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:02.575371 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:02.575375 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:02.575545 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:02.575972 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:02.575975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:02.575977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:02.576286 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:02.576456 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:02.576906 (+ 450us) server_negotiation.cc:299] Negotiation successful
0807 11:29:02.577149 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 11:29:02.578549 8650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.563026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60960 (local address 127.16.197.62:33131)
0807 11:29:02.563189 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:02.563196 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:02.563217 (+ 21us) server_negotiation.cc:407] Connection header received
0807 11:29:02.563420 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:02.563426 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:02.563504 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:02.563606 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:02.565138 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.565823 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.567037 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.567462 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.567605 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:02.572184 (+ 4579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:02.572198 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:02.572201 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:02.572233 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:02.575436 (+ 3203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:02.576276 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:02.576281 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:02.576284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:02.576461 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:02.576882 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:02.576887 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:02.576891 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:02.577248 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:02.577838 (+ 590us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:02.578011 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 11:29:02.578243 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 11:29:02.578716 8646 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 11:29:02 dist-test-slave-2w3w krb5kdc[8407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102142, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.60@KRBTEST.COM for kudu/127.16.197.60@KRBTEST.COM
Aug 07 11:29:02 dist-test-slave-2w3w krb5kdc[8407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.61@KRBTEST.COM
I20260807 11:29:02.593765 8649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.579595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35746 (local address 127.16.197.60:35135)
0807 11:29:02.579766 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:02.579773 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:02.579790 (+ 17us) server_negotiation.cc:407] Connection header received
0807 11:29:02.579936 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:02.579941 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:02.580007 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:02.580101 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:02.580901 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.581825 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.583344 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.583901 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.584045 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:02.587640 (+ 3595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:02.587649 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:02.587655 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:02.587694 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:02.590726 (+ 3032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:02.591567 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:02.591573 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:02.591578 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:02.591753 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:02.592206 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:02.592210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:02.592212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:02.592532 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:02.592686 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:02.593155 (+ 469us) server_negotiation.cc:299] Negotiation successful
0807 11:29:02.593519 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":185,"server-negotiator.queue_time_us":37}
I20260807 11:29:02.593849 8651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.579511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:35135 (local address 127.0.0.1:35746)
0807 11:29:02.579685 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:02.579707 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 11:29:02.579808 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:29:02.580116 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:29:02.580120 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:29:02.580625 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:29:02.580772 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:02.580781 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.581942 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.581949 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:02.583210 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:02.583225 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.584049 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.584053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:02.584232 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:02.584846 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:29:02.584863 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:29:02.587430 (+ 2567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:29:02.590853 (+ 3423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:02.590859 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:02.590864 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:02.591425 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:02.591902 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:02.591907 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:02.591912 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:02.592086 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:02.592676 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:29:02.592682 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:29:02.592857 (+ 175us) client_negotiation.cc:769] Sending connection context
0807 11:29:02.593044 (+ 187us) client_negotiation.cc:240] Negotiation successful
0807 11:29:02.593378 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":3}
I20260807 11:29:02.596915 8573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.580434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60402 (local address 127.16.197.61:34783)
0807 11:29:02.581032 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:02.581038 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:02.581052 (+ 14us) server_negotiation.cc:407] Connection header received
0807 11:29:02.581095 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:02.581099 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:02.581164 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:02.581259 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:02.582435 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.583531 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.584975 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.585646 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.585775 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:02.590822 (+ 5047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:02.590830 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:02.590834 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:02.590885 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:02.594290 (+ 3405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:02.595062 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:02.595067 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:02.595070 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:02.595237 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:02.595677 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:02.595680 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:02.595682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:02.595955 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:02.596149 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:02.596497 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 11:29:02.596664 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480}
I20260807 11:29:02.599202 8490 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } attempt: 4
I20260807 11:29:02.599984 8646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33df89f012a6459d9926e22163ad6af2: Bootstrap starting.
I20260807 11:29:02.603008 17172 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 11:29:02.604071 8646 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33df89f012a6459d9926e22163ad6af2: Neither blocks nor log segments found. Creating new log.
I20260807 11:29:02.605484 8646 log.cc:824] T 00000000000000000000000000000000 P 33df89f012a6459d9926e22163ad6af2: Log is configured to *not* fsync() on all Append() calls
Aug 07 11:29:02 dist-test-slave-2w3w krb5kdc[8407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102141, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for kudu/127.16.197.60@KRBTEST.COM
I20260807 11:29:02.609735 8646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33df89f012a6459d9926e22163ad6af2: No bootstrap required, opened a new log
I20260807 11:29:02.606246 8463 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2ad7ae4b264f45589313e987fadb8b99"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60960
I20260807 11:29:02.617628 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.599892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:35135 (local address 127.0.0.1:35754)
0807 11:29:02.600169 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:02.600190 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 11:29:02.600301 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:29:02.601029 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:29:02.601036 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:29:02.601440 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:29:02.601592 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:02.601601 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.602983 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.602992 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:02.604549 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:02.604572 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.605720 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.605726 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:02.605971 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:02.606840 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:29:02.606866 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:29:02.610042 (+ 3176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:29:02.614577 (+ 4535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:02.614583 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:02.614588 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:02.615193 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:02.615628 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:02.615634 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:02.615638 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:02.615811 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:02.616546 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:29:02.616552 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:29:02.616735 (+ 183us) client_negotiation.cc:769] Sending connection context
0807 11:29:02.616883 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 11:29:02.617189 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139}
I20260807 11:29:02.617647 8649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.600062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35754 (local address 127.16.197.60:35135)
0807 11:29:02.600743 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:02.600750 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:02.600767 (+ 17us) server_negotiation.cc:407] Connection header received
0807 11:29:02.600815 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:02.600825 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:02.600883 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:02.600989 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:02.601756 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.602786 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.604737 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.605527 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.606360 (+ 833us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:02.610378 (+ 4018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:02.610385 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:02.610390 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:02.610427 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:02.614412 (+ 3985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:02.615317 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:02.615322 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:02.615326 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:02.615500 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:02.615930 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:02.615934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:02.615938 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:02.616405 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:02.616563 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:02.617011 (+ 448us) server_negotiation.cc:299] Negotiation successful
0807 11:29:02.617246 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514}
I20260807 11:29:02.619395 8646 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33df89f012a6459d9926e22163ad6af2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7ae4b264f45589313e987fadb8b99" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33131 } } peers { permanent_uuid: "2f108e02dc194b7c94891ce8b033cb4e" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 34783 } } peers { permanent_uuid: "33df89f012a6459d9926e22163ad6af2" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } }
I20260807 11:29:02.619745 8646 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33df89f012a6459d9926e22163ad6af2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33df89f012a6459d9926e22163ad6af2, State: Initialized, Role: FOLLOWER
I20260807 11:29:02.620220 8646 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33df89f012a6459d9926e22163ad6af2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7ae4b264f45589313e987fadb8b99" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33131 } } peers { permanent_uuid: "2f108e02dc194b7c94891ce8b033cb4e" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 34783 } } peers { permanent_uuid: "33df89f012a6459d9926e22163ad6af2" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } }
I20260807 11:29:02.621532 8653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33df89f012a6459d9926e22163ad6af2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7ae4b264f45589313e987fadb8b99" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33131 } } peers { permanent_uuid: "2f108e02dc194b7c94891ce8b033cb4e" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 34783 } } peers { permanent_uuid: "33df89f012a6459d9926e22163ad6af2" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } } }
I20260807 11:29:02.622205 8646 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33df89f012a6459d9926e22163ad6af2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:29:02.622336 8653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33df89f012a6459d9926e22163ad6af2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 11:29:02.623950 8490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ad7ae4b264f45589313e987fadb8b99: Bootstrap starting.
I20260807 11:29:02.628307 8490 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ad7ae4b264f45589313e987fadb8b99: Neither blocks nor log segments found. Creating new log.
I20260807 11:29:02.630550 8490 log.cc:824] T 00000000000000000000000000000000 P 2ad7ae4b264f45589313e987fadb8b99: Log is configured to *not* fsync() on all Append() calls
W20260807 11:29:02.634093 8666 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 33df89f012a6459d9926e22163ad6af2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 11:29:02.634229 8666 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 11:29:02.634455 8490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ad7ae4b264f45589313e987fadb8b99: No bootstrap required, opened a new log
I20260807 11:29:02.639937 8490 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ad7ae4b264f45589313e987fadb8b99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7ae4b264f45589313e987fadb8b99" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33131 } } peers { permanent_uuid: "2f108e02dc194b7c94891ce8b033cb4e" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 34783 } } peers { permanent_uuid: "33df89f012a6459d9926e22163ad6af2" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } }
I20260807 11:29:02.640369 8490 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ad7ae4b264f45589313e987fadb8b99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ad7ae4b264f45589313e987fadb8b99, State: Initialized, Role: FOLLOWER
I20260807 11:29:02.640862 8490 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ad7ae4b264f45589313e987fadb8b99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7ae4b264f45589313e987fadb8b99" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33131 } } peers { permanent_uuid: "2f108e02dc194b7c94891ce8b033cb4e" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 34783 } } peers { permanent_uuid: "33df89f012a6459d9926e22163ad6af2" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } }
I20260807 11:29:02.630224 8576 master_runner.cc:186] Error getting master registration for 127.16.197.62:33131: OK, instance_id { permanent_uuid: "2ad7ae4b264f45589313e987fadb8b99" instance_seqno: 1786102141457284 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 11:29:02.641343 8576 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 11:29:02.641966 8669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ad7ae4b264f45589313e987fadb8b99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7ae4b264f45589313e987fadb8b99" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33131 } } peers { permanent_uuid: "2f108e02dc194b7c94891ce8b033cb4e" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 34783 } } peers { permanent_uuid: "33df89f012a6459d9926e22163ad6af2" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } } }
I20260807 11:29:02.642408 8669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ad7ae4b264f45589313e987fadb8b99 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 11:29:02.643321 8490 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ad7ae4b264f45589313e987fadb8b99 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:29:02.643445 8463 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ad7ae4b264f45589313e987fadb8b99 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 11:29:02.643671 8463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ad7ae4b264f45589313e987fadb8b99 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:29:02.647001 8463 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ad7ae4b264f45589313e987fadb8b99 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7ae4b264f45589313e987fadb8b99" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33131 } } peers { permanent_uuid: "2f108e02dc194b7c94891ce8b033cb4e" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 34783 } } peers { permanent_uuid: "33df89f012a6459d9926e22163ad6af2" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } }
I20260807 11:29:02.648573 8463 leader_election.cc:288] T 00000000000000000000000000000000 P 2ad7ae4b264f45589313e987fadb8b99 [CANDIDATE]: Term 1 election: Requested vote from peers 2f108e02dc194b7c94891ce8b033cb4e (127.16.197.61:34783), 33df89f012a6459d9926e22163ad6af2 (127.16.197.60:35135)
I20260807 11:29:02.649463 8619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ad7ae4b264f45589313e987fadb8b99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "33df89f012a6459d9926e22163ad6af2"
I20260807 11:29:02.649953 8619 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33df89f012a6459d9926e22163ad6af2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:29:02.653504 8619 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 33df89f012a6459d9926e22163ad6af2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ad7ae4b264f45589313e987fadb8b99 in term 1.
I20260807 11:29:02.654300 8438 leader_election.cc:302] T 00000000000000000000000000000000 P 2ad7ae4b264f45589313e987fadb8b99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ad7ae4b264f45589313e987fadb8b99, 33df89f012a6459d9926e22163ad6af2; no voters:
I20260807 11:29:02.654896 8669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ad7ae4b264f45589313e987fadb8b99 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:29:02.660357 8669 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ad7ae4b264f45589313e987fadb8b99 [term 1 LEADER]: Becoming Leader. State: Replica: 2ad7ae4b264f45589313e987fadb8b99, State: Running, Role: LEADER
I20260807 11:29:02.661046 8669 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ad7ae4b264f45589313e987fadb8b99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7ae4b264f45589313e987fadb8b99" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33131 } } peers { permanent_uuid: "2f108e02dc194b7c94891ce8b033cb4e" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 34783 } } peers { permanent_uuid: "33df89f012a6459d9926e22163ad6af2" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } }
I20260807 11:29:02.672655 8675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ad7ae4b264f45589313e987fadb8b99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ad7ae4b264f45589313e987fadb8b99. Latest consensus state: current_term: 1 leader_uuid: "2ad7ae4b264f45589313e987fadb8b99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7ae4b264f45589313e987fadb8b99" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33131 } } peers { permanent_uuid: "2f108e02dc194b7c94891ce8b033cb4e" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 34783 } } peers { permanent_uuid: "33df89f012a6459d9926e22163ad6af2" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } } }
I20260807 11:29:02.673208 8675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ad7ae4b264f45589313e987fadb8b99 [sys.catalog]: This master's current role is: LEADER
I20260807 11:29:02.679225 8683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 11:29:02.695622 8683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260807 11:29:02.696424 8441 tablet.cc:2369] T 00000000000000000000000000000000 P 2ad7ae4b264f45589313e987fadb8b99: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:29:02.723011 8567 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } attempt: 5
I20260807 11:29:02.724471 8573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.684538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60414 (local address 127.16.197.61:34783)
0807 11:29:02.684703 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:02.684709 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:02.687043 (+ 2334us) server_negotiation.cc:407] Connection header received
0807 11:29:02.688072 (+ 1029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:02.688077 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:02.688167 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:02.688248 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:02.693225 (+ 4977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.694345 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.698321 (+ 3976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.699048 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.699156 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:02.715533 (+ 16377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:02.715541 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:02.715546 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:02.715591 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:02.719371 (+ 3780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:02.721235 (+ 1864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:02.721240 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:02.721244 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:02.721429 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:02.722264 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:02.722269 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:02.722272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:02.722679 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:02.722904 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:02.723830 (+ 926us) server_negotiation.cc:299] Negotiation successful
0807 11:29:02.724078 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 11:29:02.722932 8649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.689009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35762 (local address 127.16.197.60:35135)
0807 11:29:02.689174 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:02.689180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:02.693069 (+ 3889us) server_negotiation.cc:407] Connection header received
0807 11:29:02.699819 (+ 6750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:02.699828 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:02.699978 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:02.700059 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:02.702338 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.703521 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.710265 (+ 6744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.711127 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.711240 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:02.715036 (+ 3796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:02.715047 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:02.715054 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:02.715111 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:02.719133 (+ 4022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:02.720119 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:02.720124 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:02.720129 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:02.720316 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:02.720796 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:02.720804 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:02.720810 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:02.721275 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:02.721530 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:02.722202 (+ 672us) server_negotiation.cc:299] Negotiation successful
0807 11:29:02.722457 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 11:29:02.734256 8650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.677488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60964 (local address 127.16.197.62:33131)
0807 11:29:02.677730 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:02.677739 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:02.683894 (+ 6155us) server_negotiation.cc:407] Connection header received
0807 11:29:02.685523 (+ 1629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:02.685532 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:02.685649 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:02.685750 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:02.697413 (+ 11663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.698715 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.701470 (+ 2755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.702180 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.702283 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:02.709926 (+ 7643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:02.709934 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:02.709939 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:02.709987 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:02.714157 (+ 4170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:02.731168 (+ 17011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:02.731175 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:02.731180 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:02.731368 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:02.731874 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:02.731879 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:02.731883 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:02.732298 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:02.732629 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:02.733359 (+ 730us) server_negotiation.cc:299] Negotiation successful
0807 11:29:02.733718 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 11:29:02.741289 8619 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 33df89f012a6459d9926e22163ad6af2 [term 1 FOLLOWER]: Refusing update from remote peer 2ad7ae4b264f45589313e987fadb8b99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:29:02.744395 8675 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2ad7ae4b264f45589313e987fadb8b99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33df89f012a6459d9926e22163ad6af2" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Aug 07 11:29:02 dist-test-slave-2w3w krb5kdc[8407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102141, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.61@KRBTEST.COM for kudu/127.16.197.60@KRBTEST.COM
W20260807 11:29:02.756803 8597 tablet.cc:2369] T 00000000000000000000000000000000 P 33df89f012a6459d9926e22163ad6af2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:29:02.763633 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.723719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:35135 (local address 127.0.0.1:35770)
0807 11:29:02.724237 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:02.724256 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 11:29:02.724352 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:29:02.727851 (+ 3499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:29:02.727857 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:29:02.728374 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:29:02.728628 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:02.728642 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.734205 (+ 5563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.734214 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:02.735647 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:02.735664 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.738270 (+ 2606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.738276 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:02.738551 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:02.748361 (+ 9810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:29:02.748400 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:29:02.755869 (+ 7469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:29:02.760446 (+ 4577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:02.760452 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:02.760456 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:02.761088 (+ 632us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:02.761588 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:02.761593 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:02.761596 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:02.761779 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:02.762514 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:29:02.762522 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:29:02.762705 (+ 183us) client_negotiation.cc:769] Sending connection context
0807 11:29:02.762878 (+ 173us) client_negotiation.cc:240] Negotiation successful
0807 11:29:02.763230 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390}
I20260807 11:29:02.764720 8693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:02.724554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35770 (local address 127.16.197.60:35135)
0807 11:29:02.727509 (+ 2955us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:02.727514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:02.727535 (+ 21us) server_negotiation.cc:407] Connection header received
0807 11:29:02.727585 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:02.727590 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:02.727660 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:02.730622 (+ 2962us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:02.730902 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.732229 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.737307 (+ 5078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:02.738075 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:02.755440 (+ 17365us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:02.756157 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:02.756164 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:02.756168 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:02.756213 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:02.760275 (+ 4062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:02.761264 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:02.761268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:02.761271 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:02.761440 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:02.761951 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:02.761960 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:02.761963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:02.762365 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:02.763903 (+ 1538us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:02.764047 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 11:29:02.764290 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2801,"thread_start_us":2602,"threads_started":1}
I20260807 11:29:02.772033 8567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f108e02dc194b7c94891ce8b033cb4e: Bootstrap starting.
I20260807 11:29:02.764122 8420 master_runner.cc:186] Error getting master registration for 127.16.197.61:34783: OK, instance_id { permanent_uuid: "2f108e02dc194b7c94891ce8b033cb4e" instance_seqno: 1786102141985267 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 11:29:02.776616 8420 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 11:29:02.779882 8567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f108e02dc194b7c94891ce8b033cb4e: Neither blocks nor log segments found. Creating new log.
I20260807 11:29:02.781564 8567 log.cc:824] T 00000000000000000000000000000000 P 2f108e02dc194b7c94891ce8b033cb4e: Log is configured to *not* fsync() on all Append() calls
I20260807 11:29:02.798552 8567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f108e02dc194b7c94891ce8b033cb4e: No bootstrap required, opened a new log
I20260807 11:29:02.811846 8567 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f108e02dc194b7c94891ce8b033cb4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7ae4b264f45589313e987fadb8b99" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33131 } } peers { permanent_uuid: "2f108e02dc194b7c94891ce8b033cb4e" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 34783 } } peers { permanent_uuid: "33df89f012a6459d9926e22163ad6af2" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } }
I20260807 11:29:02.812340 8567 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f108e02dc194b7c94891ce8b033cb4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f108e02dc194b7c94891ce8b033cb4e, State: Initialized, Role: FOLLOWER
I20260807 11:29:02.813036 8567 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f108e02dc194b7c94891ce8b033cb4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7ae4b264f45589313e987fadb8b99" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33131 } } peers { permanent_uuid: "2f108e02dc194b7c94891ce8b033cb4e" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 34783 } } peers { permanent_uuid: "33df89f012a6459d9926e22163ad6af2" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } }
I20260807 11:29:02.814194 8698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f108e02dc194b7c94891ce8b033cb4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7ae4b264f45589313e987fadb8b99" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33131 } } peers { permanent_uuid: "2f108e02dc194b7c94891ce8b033cb4e" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 34783 } } peers { permanent_uuid: "33df89f012a6459d9926e22163ad6af2" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } } }
I20260807 11:29:02.814636 8698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f108e02dc194b7c94891ce8b033cb4e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 11:29:02.815862 8567 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f108e02dc194b7c94891ce8b033cb4e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:29:02.815929 8542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ad7ae4b264f45589313e987fadb8b99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2f108e02dc194b7c94891ce8b033cb4e"
I20260807 11:29:02.816316 8541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f108e02dc194b7c94891ce8b033cb4e [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:29:02.819490 8541 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2f108e02dc194b7c94891ce8b033cb4e [term 1 FOLLOWER]: Refusing update from remote peer 2ad7ae4b264f45589313e987fadb8b99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:29:02.820117 8653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33df89f012a6459d9926e22163ad6af2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ad7ae4b264f45589313e987fadb8b99. Latest consensus state: current_term: 1 leader_uuid: "2ad7ae4b264f45589313e987fadb8b99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7ae4b264f45589313e987fadb8b99" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33131 } } peers { permanent_uuid: "2f108e02dc194b7c94891ce8b033cb4e" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 34783 } } peers { permanent_uuid: "33df89f012a6459d9926e22163ad6af2" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } } }
I20260807 11:29:02.820458 8653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33df89f012a6459d9926e22163ad6af2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 11:29:02.823805 8542 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2f108e02dc194b7c94891ce8b033cb4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ad7ae4b264f45589313e987fadb8b99 in term 1.
I20260807 11:29:02.825920 8691 mvcc.cc:205] Tried to move back new op lower bound from 7315874376647663616 to 7315874376346689536. Current Snapshot: MvccSnapshot[applied={T|T < 7315874376647663616}]
I20260807 11:29:02.827792 8695 mvcc.cc:205] Tried to move back new op lower bound from 7315874376647663616 to 7315874376346689536. Current Snapshot: MvccSnapshot[applied={T|T < 7315874376647663616}]
I20260807 11:29:02.831228 8653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33df89f012a6459d9926e22163ad6af2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2ad7ae4b264f45589313e987fadb8b99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7ae4b264f45589313e987fadb8b99" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33131 } } peers { permanent_uuid: "2f108e02dc194b7c94891ce8b033cb4e" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 34783 } } peers { permanent_uuid: "33df89f012a6459d9926e22163ad6af2" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } } }
I20260807 11:29:02.831516 8653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33df89f012a6459d9926e22163ad6af2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 11:29:02.836691 8669 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2ad7ae4b264f45589313e987fadb8b99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f108e02dc194b7c94891ce8b033cb4e" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 34783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:29:02.847898 8669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ad7ae4b264f45589313e987fadb8b99 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2ad7ae4b264f45589313e987fadb8b99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7ae4b264f45589313e987fadb8b99" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33131 } } peers { permanent_uuid: "2f108e02dc194b7c94891ce8b033cb4e" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 34783 } } peers { permanent_uuid: "33df89f012a6459d9926e22163ad6af2" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } } }
I20260807 11:29:02.848272 8669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ad7ae4b264f45589313e987fadb8b99 [sys.catalog]: This master's current role is: LEADER
I20260807 11:29:02.849316 8683 catalog_manager.cc:1347] Generated new cluster ID: 81a5064a482c4c228fcc2c749cc59aee
I20260807 11:29:02.849457 8683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:29:02.871946 8683 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:29:02.874002 8683 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:29:02.876513 8698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f108e02dc194b7c94891ce8b033cb4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ad7ae4b264f45589313e987fadb8b99. Latest consensus state: current_term: 1 leader_uuid: "2ad7ae4b264f45589313e987fadb8b99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7ae4b264f45589313e987fadb8b99" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33131 } } peers { permanent_uuid: "2f108e02dc194b7c94891ce8b033cb4e" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 34783 } } peers { permanent_uuid: "33df89f012a6459d9926e22163ad6af2" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } } }
I20260807 11:29:02.876845 8698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f108e02dc194b7c94891ce8b033cb4e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 11:29:02.877391 8708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f108e02dc194b7c94891ce8b033cb4e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2ad7ae4b264f45589313e987fadb8b99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7ae4b264f45589313e987fadb8b99" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33131 } } peers { permanent_uuid: "2f108e02dc194b7c94891ce8b033cb4e" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 34783 } } peers { permanent_uuid: "33df89f012a6459d9926e22163ad6af2" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } } }
I20260807 11:29:02.878108 8708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f108e02dc194b7c94891ce8b033cb4e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 11:29:02.879392 8669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ad7ae4b264f45589313e987fadb8b99 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2ad7ae4b264f45589313e987fadb8b99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad7ae4b264f45589313e987fadb8b99" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33131 } } peers { permanent_uuid: "2f108e02dc194b7c94891ce8b033cb4e" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 34783 } } peers { permanent_uuid: "33df89f012a6459d9926e22163ad6af2" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 35135 } } }
I20260807 11:29:02.879668 8669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ad7ae4b264f45589313e987fadb8b99 [sys.catalog]: This master's current role is: LEADER
I20260807 11:29:02.908668 8683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ad7ae4b264f45589313e987fadb8b99: Generated new TSK 0
I20260807 11:29:02.909953 8683 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:29:02.942191 8716 catalog_manager.cc:1259] Loaded cluster ID: 81a5064a482c4c228fcc2c749cc59aee
I20260807 11:29:02.942317 8716 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2f108e02dc194b7c94891ce8b033cb4e: loading cluster ID for follower catalog manager: success
I20260807 11:29:02.946128 8716 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2f108e02dc194b7c94891ce8b033cb4e: acquiring CA information for follower catalog manager: success
I20260807 11:29:02.951538 8716 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2f108e02dc194b7c94891ce8b033cb4e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 11:29:03.067785 8650 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:29:03.064977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60976 (local address 127.16.197.62:33131)
0807 11:29:03.065230 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:03.065241 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:03.065721 (+ 480us) server_negotiation.cc:407] Connection header received
0807 11:29:03.066038 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:03.066045 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:03.066136 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:03.066212 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:03.067502 (+ 1290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60976: BlockingRecv error: recv got EOF from 127.0.0.1:60976 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 11:29:03.070591 8573 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:29:03.066242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60422 (local address 127.16.197.61:34783)
0807 11:29:03.066434 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:03.066440 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:03.068584 (+ 2144us) server_negotiation.cc:407] Connection header received
0807 11:29:03.068952 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:03.068958 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:03.069057 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:03.069131 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:03.070351 (+ 1220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60422: BlockingRecv error: recv got EOF from 127.0.0.1:60422 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260807 11:29:03.074265 8693 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:29:03.071953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35780 (local address 127.16.197.60:35135)
0807 11:29:03.072129 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:03.072135 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:03.072322 (+ 187us) server_negotiation.cc:407] Connection header received
0807 11:29:03.072582 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:03.072588 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:03.072749 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:03.072831 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:03.074011 (+ 1180us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35780: BlockingRecv error: recv got EOF from 127.0.0.1:35780 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 11:29:03.133975 8573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:03.117006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60432 (local address 127.16.197.61:34783)
0807 11:29:03.117215 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:03.117222 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:03.125616 (+ 8394us) server_negotiation.cc:407] Connection header received
0807 11:29:03.126086 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:03.126092 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:03.126346 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:03.126499 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:29:03.127374 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:03.128514 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:03.130647 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:03.131394 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:03.131501 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:03.132317 (+ 816us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:29:03.133083 (+ 766us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:29:03.133218 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:03.133474 (+ 256us) server_negotiation.cc:299] Negotiation successful
0807 11:29:03.133589 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260807 11:29:03.180238 8693 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:29:03.169519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35786 (local address 127.16.197.60:35135)
0807 11:29:03.169754 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:03.169761 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:03.177910 (+ 8149us) server_negotiation.cc:407] Connection header received
0807 11:29:03.178477 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:03.178484 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:03.178627 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:03.178712 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:03.179919 (+ 1207us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35786: BlockingRecv error: recv got EOF from 127.0.0.1:35786 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 11:29:03.227137 8573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:03.211740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60436 (local address 127.16.197.61:34783)
0807 11:29:03.211947 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:03.211954 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:03.219490 (+ 7536us) server_negotiation.cc:407] Connection header received
0807 11:29:03.220069 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:03.220076 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:03.220202 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:03.220279 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:29:03.221064 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:03.222305 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:03.224631 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:03.225396 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:03.225506 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:03.226196 (+ 690us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:29:03.226366 (+ 170us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:29:03.226434 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:03.226674 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 11:29:03.226768 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260807 11:29:03.271418 8693 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:29:03.261213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35796 (local address 127.16.197.60:35135)
0807 11:29:03.261464 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:03.261471 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:03.269013 (+ 7542us) server_negotiation.cc:407] Connection header received
0807 11:29:03.269601 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:03.269608 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:03.269717 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:03.269809 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:03.271148 (+ 1339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35796: BlockingRecv error: recv got EOF from 127.0.0.1:35796 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
I20260807 11:29:03.325265 8573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:03.309826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60446 (local address 127.16.197.61:34783)
0807 11:29:03.310043 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:03.310053 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:03.317657 (+ 7604us) server_negotiation.cc:407] Connection header received
0807 11:29:03.318213 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:03.318220 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:03.318336 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:03.318411 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:29:03.319253 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:03.320489 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:03.322609 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:03.323399 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:03.323540 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:03.324237 (+ 697us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:29:03.324410 (+ 173us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:29:03.324484 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:03.324817 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 11:29:03.324907 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260807 11:29:03.353247 8693 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:29:03.344512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35812 (local address 127.16.197.60:35135)
0807 11:29:03.344708 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:03.344715 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:03.351545 (+ 6830us) server_negotiation.cc:407] Connection header received
0807 11:29:03.351735 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:03.351741 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:03.351854 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:03.351990 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:03.352982 (+ 992us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35812: BlockingRecv error: recv got EOF from 127.0.0.1:35812 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 11:29:03.369138 8573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:03.361785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60456 (local address 127.16.197.61:34783)
0807 11:29:03.362028 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:03.362035 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:03.362448 (+ 413us) server_negotiation.cc:407] Connection header received
0807 11:29:03.362669 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:03.362677 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:03.362777 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:03.362885 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:29:03.363471 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:03.364672 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:03.366663 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:03.367467 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:03.367615 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:03.368174 (+ 559us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:29:03.368318 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:29:03.368403 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:03.368610 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 11:29:03.368699 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
W20260807 11:29:03.379006 8693 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:29:03.376591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35814 (local address 127.16.197.60:35135)
0807 11:29:03.376841 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:03.376848 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:03.377464 (+ 616us) server_negotiation.cc:407] Connection header received
0807 11:29:03.377612 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:03.377619 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:03.377725 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:03.377849 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:03.378728 (+ 879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35814: BlockingRecv error: recv got EOF from 127.0.0.1:35814 (error 108)
Metrics: {"server-negotiator.queue_time_us":134}
I20260807 11:29:03.399180 8573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:03.391465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60472 (local address 127.16.197.61:34783)
0807 11:29:03.391682 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:03.391691 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:03.392327 (+ 636us) server_negotiation.cc:407] Connection header received
0807 11:29:03.392557 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:03.392563 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:03.392672 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:03.392797 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:29:03.393319 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:03.394502 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:03.396812 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:03.397581 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:03.397739 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:03.398237 (+ 498us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:29:03.398427 (+ 190us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:29:03.398529 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:03.398691 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 11:29:03.398779 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260807 11:29:03.408836 8693 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:29:03.406461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35824 (local address 127.16.197.60:35135)
0807 11:29:03.406665 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:03.406671 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:03.407253 (+ 582us) server_negotiation.cc:407] Connection header received
0807 11:29:03.407452 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:03.407459 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:03.407572 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:03.407690 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:03.408614 (+ 924us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35824: BlockingRecv error: recv got EOF from 127.0.0.1:35824 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 11:29:03.435642 8573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:03.429564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60486 (local address 127.16.197.61:34783)
0807 11:29:03.429763 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:03.429770 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:03.430118 (+ 348us) server_negotiation.cc:407] Connection header received
0807 11:29:03.430286 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:03.430292 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:03.430410 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:03.430525 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:29:03.430915 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:03.432175 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:03.433520 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:03.434224 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:03.434375 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:03.434761 (+ 386us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:29:03.434907 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:29:03.435089 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:03.435208 (+ 119us) server_negotiation.cc:299] Negotiation successful
0807 11:29:03.435302 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260807 11:29:03.444082 8693 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:29:03.441845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35836 (local address 127.16.197.60:35135)
0807 11:29:03.442063 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:03.442072 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:03.442468 (+ 396us) server_negotiation.cc:407] Connection header received
0807 11:29:03.442687 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:03.442697 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:03.442767 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:03.442881 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:03.443809 (+ 928us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35836: BlockingRecv error: recv got EOF from 127.0.0.1:35836 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 11:29:03.487875 8573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:03.482426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60492 (local address 127.16.197.61:34783)
0807 11:29:03.482628 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:03.482635 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:03.483043 (+ 408us) server_negotiation.cc:407] Connection header received
0807 11:29:03.483255 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:03.483261 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:03.483377 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:03.483505 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:29:03.483924 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:03.484981 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:03.486287 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:03.486711 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:03.486806 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:03.487261 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:29:03.487385 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:29:03.487455 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:03.487576 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 11:29:03.487639 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260807 11:29:03.495385 8693 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:29:03.493533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35850 (local address 127.16.197.60:35135)
0807 11:29:03.493791 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:03.493797 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:03.494064 (+ 267us) server_negotiation.cc:407] Connection header received
0807 11:29:03.494159 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:03.494163 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:03.494252 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:03.494343 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:03.495139 (+ 796us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35850: BlockingRecv error: recv got EOF from 127.0.0.1:35850 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 11:29:03.569561 8573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:03.562669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60496 (local address 127.16.197.61:34783)
0807 11:29:03.562952 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:03.563013 (+ 61us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:03.563224 (+ 211us) server_negotiation.cc:407] Connection header received
0807 11:29:03.563392 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:03.563396 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:03.563464 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:03.563550 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:29:03.564089 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:03.565320 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:03.567579 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:03.568362 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:03.568481 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:03.568971 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:29:03.569088 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:29:03.569167 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:03.569273 (+ 106us) server_negotiation.cc:299] Negotiation successful
0807 11:29:03.569340 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
W20260807 11:29:03.577507 8693 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:29:03.575790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35854 (local address 127.16.197.60:35135)
0807 11:29:03.576029 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:03.576036 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:03.576325 (+ 289us) server_negotiation.cc:407] Connection header received
0807 11:29:03.576467 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:03.576471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:03.576541 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:03.576631 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:03.577289 (+ 658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35854: BlockingRecv error: recv got EOF from 127.0.0.1:35854 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 11:29:03.636476 8666 catalog_manager.cc:1259] Loaded cluster ID: 81a5064a482c4c228fcc2c749cc59aee
I20260807 11:29:03.636579 8666 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 33df89f012a6459d9926e22163ad6af2: loading cluster ID for follower catalog manager: success
I20260807 11:29:03.638944 8666 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 33df89f012a6459d9926e22163ad6af2: acquiring CA information for follower catalog manager: success
I20260807 11:29:03.640695 8666 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 33df89f012a6459d9926e22163ad6af2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 11:29:03.714911 8573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:03.709307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60510 (local address 127.16.197.61:34783)
0807 11:29:03.709537 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:03.709546 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:03.709979 (+ 433us) server_negotiation.cc:407] Connection header received
0807 11:29:03.710226 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:03.710230 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:03.710302 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:03.710418 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:29:03.710898 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:03.711681 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:03.713116 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:03.713547 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:03.713693 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:03.714132 (+ 439us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:29:03.714252 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:29:03.714355 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:03.714520 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 11:29:03.714608 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 11:29:03.726668 8693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:03.720884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35860 (local address 127.16.197.60:35135)
0807 11:29:03.721127 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:03.721134 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:03.721416 (+ 282us) server_negotiation.cc:407] Connection header received
0807 11:29:03.721561 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:03.721565 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:03.721720 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:03.721838 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:29:03.722195 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:03.723032 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:03.724364 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:03.724857 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:03.724957 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:03.725416 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:29:03.726020 (+ 604us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:29:03.726145 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:03.726270 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 11:29:03.726339 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 11:29:03.740146 8826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:03.733004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60992 (local address 127.16.197.62:33131)
0807 11:29:03.733753 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:03.733763 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:03.733789 (+ 26us) server_negotiation.cc:407] Connection header received
0807 11:29:03.733946 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:03.733953 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:03.734172 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:03.734403 (+ 231us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:29:03.734953 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:03.735908 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:03.737937 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:03.738335 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:03.738477 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:03.738940 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:29:03.739321 (+ 381us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:29:03.739461 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:03.739695 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 11:29:03.739796 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":253,"threads_started":1}
I20260807 11:29:03.741416 8693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:03.734324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35866 (local address 127.16.197.60:35135)
0807 11:29:03.734495 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:03.734500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:03.735145 (+ 645us) server_negotiation.cc:407] Connection header received
0807 11:29:03.735339 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:03.735345 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:03.735419 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:03.735514 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:29:03.736194 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:03.737310 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:03.739462 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:03.740023 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:03.740154 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:03.740644 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:29:03.740814 (+ 170us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:29:03.740911 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:03.741057 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 11:29:03.741143 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 11:29:03.741822 8573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:03.733714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60520 (local address 127.16.197.61:34783)
0807 11:29:03.733943 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:03.733950 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:03.734204 (+ 254us) server_negotiation.cc:407] Connection header received
0807 11:29:03.735108 (+ 904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:03.735116 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:03.735199 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:03.735329 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:29:03.736340 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:03.737075 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:03.739440 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:03.740156 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:03.740299 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:03.740903 (+ 604us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:29:03.741035 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:29:03.741120 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:03.741481 (+ 361us) server_negotiation.cc:299] Negotiation successful
0807 11:29:03.741566 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 11:29:03.745306 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 8420
I20260807 11:29:03.746865 8485 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:29:03.860961 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 8420
W20260807 11:29:03.884249 8595 connection.cc:535] server connection from 127.0.0.1:35754 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 11:29:03.884248 8518 connection.cc:535] server connection from 127.0.0.1:60386 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 11:29:03.884871 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 8498
I20260807 11:29:03.885953 8562 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:29:03.994665 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 8498
I20260807 11:29:04.017866 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 8576
I20260807 11:29:04.019151 8640 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:29:04.130260 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 8576
2026-08-07T11:29:04Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4620 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 11:29:04 dist-test-slave-2w3w krb5kdc[8835](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 11:29:04 dist-test-slave-2w3w krb5kdc[8835](info): set up 2 sockets
Aug 07 11:29:04 dist-test-slave-2w3w krb5kdc[8835](info): commencing operation
krb5kdc: starting...
W20260807 11:29:05.293162 17172 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/krb5kdc/test-user.keytab.
Aug 07 11:29:05 dist-test-slave-2w3w krb5kdc[8835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T11:29:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T11:29:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.197.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
Entry for principal kudu/127.16.197.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab.
I20260807 11:29:05.601795 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:33109
--webserver_interface=127.16.197.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:40249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:33109,127.16.197.61:46179,127.16.197.60:44381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 11:29:05.955233 8851 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 11:29:05.955624 8851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:29:05.955760 8851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:29:05.955904 8851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:29:05.963200 8851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:29:05.963348 8851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:29:05.963462 8851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:29:05.963557 8851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:29:05.963657 8851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:29:05.974227 8851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:40249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:33109,127.16.197.61:46179,127.16.197.60:44381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.62.keytab
--principal=kudu/127.16.197.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.62:33109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.197.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:29:05.975732 8851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:29:05.977115 8851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:29:05.989105 8856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:29:05.989074 8857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:29:05.989017 8859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:29:05.990556 8851 server_base.cc:1034] running on GCE node
I20260807 11:29:05.991456 8851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:29:05.992996 8851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:29:05.994240 8851 hybrid_clock.cc:648] HybridClock initialized: now 1786102145994181 us; error 44 us; skew 500 ppm
I20260807 11:29:05.994529 8851 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:29:05 dist-test-slave-2w3w krb5kdc[8835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102145, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:29:05.997799 8851 init.cc:377] Logged in from keytab as kudu/127.16.197.62@KRBTEST.COM (short username kudu)
I20260807 11:29:05.999365 8851 webserver.cc:460] Webserver started at http://127.16.197.62:45965/ using document root <none> and password file <none>
I20260807 11:29:06.000083 8851 fs_manager.cc:359] Metadata directory not provided
I20260807 11:29:06.000193 8851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:29:06.000530 8851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:29:06.003842 8851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-0/data/instance:
uuid: "5bb10dc3be6f49448bb6cced3affc0fa"
format_stamp: "Formatted at 2026-08-07 11:29:05 on dist-test-slave-2w3w"
server_key: "8d1ad00001fbeb4b5c62d02cbece3af3"
server_key_iv: "120864f91f205e460c930ac699ec6f53"
server_key_version: "encryptionkey@0"
I20260807 11:29:06.004664 8851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-0/wal/instance:
uuid: "5bb10dc3be6f49448bb6cced3affc0fa"
format_stamp: "Formatted at 2026-08-07 11:29:05 on dist-test-slave-2w3w"
server_key: "8d1ad00001fbeb4b5c62d02cbece3af3"
server_key_iv: "120864f91f205e460c930ac699ec6f53"
server_key_version: "encryptionkey@0"
I20260807 11:29:06.009965 8851 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 11:29:06.013578 8866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:29:06.015061 8851 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 11:29:06.015259 8851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-0/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-0/wal
uuid: "5bb10dc3be6f49448bb6cced3affc0fa"
format_stamp: "Formatted at 2026-08-07 11:29:05 on dist-test-slave-2w3w"
server_key: "8d1ad00001fbeb4b5c62d02cbece3af3"
server_key_iv: "120864f91f205e460c930ac699ec6f53"
server_key_version: "encryptionkey@0"
I20260807 11:29:06.015486 8851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:29:06.030717 8851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:29:06.034632 8851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:29:06.034978 8851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:29:06.054147 8851 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.62:33109
I20260807 11:29:06.054169 8921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.62:33109 every 8 connection(s)
I20260807 11:29:06.055585 8851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-0/data/info.pb
I20260807 11:29:06.059296 8922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:29:06.064293 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 8851
I20260807 11:29:06.064481 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-0/wal/instance
I20260807 11:29:06.064941 17172 external_mini_cluster.cc:1428] Setting key a730fa2a2bd1c1617648fa0694e410d9
I20260807 11:29:06.065896 8922 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33109 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 11:29:06 dist-test-slave-2w3w krb5kdc[8835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102145, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:29:06.089783 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.069042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33109 (local address 127.0.0.1:33388)
0807 11:29:06.069884 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:06.069938 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 11:29:06.072223 (+ 2285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:29:06.073067 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:29:06.073073 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:29:06.073809 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:29:06.074141 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:06.074158 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:06.075844 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:06.075860 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:06.077148 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:06.077166 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:06.078189 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:06.078195 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:06.078324 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:06.079088 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:29:06.079111 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:29:06.081263 (+ 2152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:29:06.085746 (+ 4483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:06.085756 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:06.085766 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:06.086159 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:06.086626 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:06.086629 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:06.086631 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:06.086784 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:06.087855 (+ 1071us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:29:06.087862 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:29:06.088056 (+ 194us) client_negotiation.cc:769] Sending connection context
0807 11:29:06.088272 (+ 216us) client_negotiation.cc:240] Negotiation successful
0807 11:29:06.088691 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":363,"threads_started":1}
I20260807 11:29:06.089828 8925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.069043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33388 (local address 127.16.197.62:33109)
0807 11:29:06.071128 (+ 2085us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:06.071166 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:06.071257 (+ 91us) server_negotiation.cc:407] Connection header received
0807 11:29:06.072601 (+ 1344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:06.072606 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:06.072851 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:06.073051 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:06.074370 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:06.075691 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:06.077313 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:06.077896 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:06.078114 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:06.081534 (+ 3420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:06.081547 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:06.081564 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:06.081605 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:06.085591 (+ 3986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:06.086303 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:06.086313 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:06.086316 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:06.086512 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:06.086896 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:06.086904 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:06.086910 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:06.087710 (+ 800us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:06.087945 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:06.088451 (+ 506us) server_negotiation.cc:299] Negotiation successful
0807 11:29:06.088945 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":124,"server-negotiator.queue_time_us":1703,"thread_start_us":2121,"threads_started":1}
I20260807 11:29:06.093181 8922 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.61" port: 46179 } has no permanent_uuid. Determining permanent_uuid...
I20260807 11:29:06.094494 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.094083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:46179 (local address 127.0.0.1:55780)
0807 11:29:06.094258 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:06.094378 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260807 11:29:06.094803 8870 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.197.61:46179: connect: Connection refused (error 111)
I20260807 11:29:06.096444 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.096073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:46179 (local address 127.0.0.1:55782)
0807 11:29:06.096232 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:06.096300 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 11:29:06.096650 8922 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.61:46179: Network error: Client connection negotiation failed: client connection to 127.16.197.61:46179: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.16.197.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.61.keytab.
Entry for principal kudu/127.16.197.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.61.keytab.
I20260807 11:29:06.135123 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.61:46179
--webserver_interface=127.16.197.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.61.keytab
--principal=kudu/127.16.197.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:40249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:33109,127.16.197.61:46179,127.16.197.60:44381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 11:29:06.145963 8922 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.61" port: 46179 } attempt: 1
I20260807 11:29:06.147167 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.146653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:46179 (local address 127.0.0.1:55796)
0807 11:29:06.146857 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:06.147016 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 11:29:06.148182 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.147828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:46179 (local address 127.0.0.1:55802)
0807 11:29:06.147984 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:06.148059 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 11:29:06.148386 8922 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.61:46179: Network error: Client connection negotiation failed: client connection to 127.16.197.61:46179: connect: Connection refused (error 111)
I20260807 11:29:06.216646 8922 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.61" port: 46179 } attempt: 2
I20260807 11:29:06.217860 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.217345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:46179 (local address 127.0.0.1:55814)
0807 11:29:06.217604 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:06.217714 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 11:29:06.219080 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.218609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:46179 (local address 127.0.0.1:55818)
0807 11:29:06.218746 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:06.218871 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 11:29:06.219385 8922 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.61:46179: Network error: Client connection negotiation failed: client connection to 127.16.197.61:46179: connect: Connection refused (error 111)
I20260807 11:29:06.310683 8922 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.61" port: 46179 } attempt: 3
I20260807 11:29:06.311906 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.311453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:46179 (local address 127.0.0.1:55832)
0807 11:29:06.311660 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:06.311756 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 11:29:06.312973 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.312568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:46179 (local address 127.0.0.1:55848)
0807 11:29:06.312716 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:06.312791 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 11:29:06.313171 8922 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.61:46179: Network error: Client connection negotiation failed: client connection to 127.16.197.61:46179: connect: Connection refused (error 111)
I20260807 11:29:06.456477 8922 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.61" port: 46179 } attempt: 4
I20260807 11:29:06.457716 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.457187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:46179 (local address 127.0.0.1:55856)
0807 11:29:06.457495 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:06.457585 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
I20260807 11:29:06.458779 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.458430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:46179 (local address 127.0.0.1:55872)
0807 11:29:06.458584 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:06.458658 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.61:46179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 11:29:06.459012 8922 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.61:46179: Network error: Client connection negotiation failed: client connection to 127.16.197.61:46179: connect: Connection refused (error 111)
W20260807 11:29:06.475189 8929 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 11:29:06.475577 8929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:29:06.475693 8929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:29:06.475859 8929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 11:29:06.483482 8929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:29:06.483572 8929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:29:06.483698 8929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:29:06.483769 8929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:29:06.483887 8929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:29:06.493983 8929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:40249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:33109,127.16.197.61:46179,127.16.197.60:44381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.61.keytab
--principal=kudu/127.16.197.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.61:46179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.16.197.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:29:06.495452 8929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:29:06.496806 8929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:29:06.508937 8937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:29:06.509045 8934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:29:06.509024 8935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:29:06.509779 8929 server_base.cc:1034] running on GCE node
I20260807 11:29:06.510541 8929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:29:06.511976 8929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:29:06.513226 8929 hybrid_clock.cc:648] HybridClock initialized: now 1786102146513156 us; error 44 us; skew 500 ppm
I20260807 11:29:06.513564 8929 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:29:06 dist-test-slave-2w3w krb5kdc[8835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102146, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:29:06.516706 8929 init.cc:377] Logged in from keytab as kudu/127.16.197.61@KRBTEST.COM (short username kudu)
I20260807 11:29:06.518191 8929 webserver.cc:460] Webserver started at http://127.16.197.61:46485/ using document root <none> and password file <none>
I20260807 11:29:06.518882 8929 fs_manager.cc:359] Metadata directory not provided
I20260807 11:29:06.519042 8929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:29:06.519383 8929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:29:06.522301 8929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-1/data/instance:
uuid: "e0178f812b5947b0824f6830b68f5925"
format_stamp: "Formatted at 2026-08-07 11:29:06 on dist-test-slave-2w3w"
server_key: "588d79b00e4eac57de66f1af4cd1b364"
server_key_iv: "fab5e3b74f93b6d84f770bb6be609791"
server_key_version: "encryptionkey@0"
I20260807 11:29:06.523176 8929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-1/wal/instance:
uuid: "e0178f812b5947b0824f6830b68f5925"
format_stamp: "Formatted at 2026-08-07 11:29:06 on dist-test-slave-2w3w"
server_key: "588d79b00e4eac57de66f1af4cd1b364"
server_key_iv: "fab5e3b74f93b6d84f770bb6be609791"
server_key_version: "encryptionkey@0"
I20260807 11:29:06.528133 8929 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 11:29:06.531577 8944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:29:06.532791 8929 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 11:29:06.532987 8929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-1/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-1/wal
uuid: "e0178f812b5947b0824f6830b68f5925"
format_stamp: "Formatted at 2026-08-07 11:29:06 on dist-test-slave-2w3w"
server_key: "588d79b00e4eac57de66f1af4cd1b364"
server_key_iv: "fab5e3b74f93b6d84f770bb6be609791"
server_key_version: "encryptionkey@0"
I20260807 11:29:06.533203 8929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:29:06.553936 8929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:29:06.557911 8929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:29:06.558190 8929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:29:06.576642 8929 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.61:46179
I20260807 11:29:06.576666 8999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.61:46179 every 8 connection(s)
I20260807 11:29:06.578016 8929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-1/data/info.pb
I20260807 11:29:06.581802 9000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:29:06.584717 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 8929
I20260807 11:29:06.584916 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-1/wal/instance
I20260807 11:29:06.585489 17172 external_mini_cluster.cc:1428] Setting key 72a7539a2464867df44cdb8566fb994e
I20260807 11:29:06.591162 9000 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33109 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 11:29:06 dist-test-slave-2w3w krb5kdc[8835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102146, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.61@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:29:06.615674 9003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.594607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33396 (local address 127.16.197.62:33109)
0807 11:29:06.595186 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:06.595193 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:06.596238 (+ 1045us) server_negotiation.cc:407] Connection header received
0807 11:29:06.598237 (+ 1999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:06.598247 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:06.598353 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:06.598463 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:06.600812 (+ 2349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:06.601547 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:06.604212 (+ 2665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:06.604648 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:06.604754 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:06.609368 (+ 4614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:06.609373 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:06.609376 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:06.609435 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:06.612247 (+ 2812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:06.613124 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:06.613130 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:06.613132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:06.613266 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:06.613739 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:06.613751 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:06.613753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:06.614061 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:06.614226 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:06.615039 (+ 813us) server_negotiation.cc:299] Negotiation successful
0807 11:29:06.615287 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":293,"threads_started":1}
I20260807 11:29:06.616719 9004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.595208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33109 (local address 127.0.0.1:33396)
0807 11:29:06.596025 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:06.596072 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 11:29:06.597928 (+ 1856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:29:06.598769 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:29:06.598782 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:29:06.599701 (+ 919us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:29:06.600555 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:06.600584 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:06.601729 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:06.601738 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:06.604007 (+ 2269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:06.604046 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:06.604908 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:06.604918 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:06.605313 (+ 395us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:06.606160 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:29:06.606196 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:29:06.609090 (+ 2894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:29:06.612423 (+ 3333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:06.612433 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:06.612453 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:06.612964 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:06.613389 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:06.613395 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:06.613399 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:06.613605 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:06.614204 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:29:06.614213 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:29:06.614607 (+ 394us) client_negotiation.cc:769] Sending connection context
0807 11:29:06.614899 (+ 292us) client_negotiation.cc:240] Negotiation successful
0807 11:29:06.615500 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":330,"threads_started":1}
WARNING: no policy specified for kudu/127.16.197.60@KRBTEST.COM; defaulting to no policy
I20260807 11:29:06.618621 9000 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.61" port: 46179 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.197.60@KRBTEST.COM" created.
Aug 07 11:29:06 dist-test-slave-2w3w krb5kdc[8835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102146, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.61@KRBTEST.COM for kudu/127.16.197.61@KRBTEST.COM
I20260807 11:29:06.635095 9004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.619566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:46179 (local address 127.0.0.1:55880)
0807 11:29:06.619752 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:06.619773 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 11:29:06.619916 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:29:06.620802 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:29:06.620805 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:29:06.621363 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:29:06.621589 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:06.621604 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:06.623023 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:06.623032 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:06.624502 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:06.624519 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:06.625386 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:06.625394 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:06.625624 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:06.626293 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:29:06.626323 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:29:06.628973 (+ 2650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:29:06.632531 (+ 3558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:06.632536 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:06.632539 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:06.632917 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:06.633370 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:06.633373 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:06.633375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:06.633521 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:06.634285 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:29:06.634288 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:29:06.634405 (+ 117us) client_negotiation.cc:769] Sending connection context
0807 11:29:06.634531 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 11:29:06.634758 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 11:29:06.635545 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.619594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55880 (local address 127.16.197.61:46179)
0807 11:29:06.620258 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:06.620269 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:06.620306 (+ 37us) server_negotiation.cc:407] Connection header received
0807 11:29:06.620373 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:06.620383 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:06.620646 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:06.620812 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:06.621769 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:06.622841 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:06.624658 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:06.625224 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:06.625324 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:06.629215 (+ 3891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:06.629231 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:06.629246 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:06.629277 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:06.632356 (+ 3079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:06.633060 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:06.633070 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:06.633075 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:06.633252 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:06.633633 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:06.633641 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:06.633647 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:06.634160 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:06.634391 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:06.634738 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 11:29:06.635078 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":324,"threads_started":1}
I20260807 11:29:06.638201 9000 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } has no permanent_uuid. Determining permanent_uuid...
I20260807 11:29:06.639503 9004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.639035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:44381 (local address 127.0.0.1:44844)
0807 11:29:06.639223 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:06.639344 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 11:29:06.639782 8947 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.197.60:44381: connect: Connection refused (error 111)
I20260807 11:29:06.641446 9004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.641093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:44381 (local address 127.0.0.1:44846)
0807 11:29:06.641251 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:06.641332 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 11:29:06.641680 9000 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.60:44381: Network error: Client connection negotiation failed: client connection to 127.16.197.60:44381: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.197.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.60.keytab.
Entry for principal kudu/127.16.197.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.60.keytab.
I20260807 11:29:06.658432 17172 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRFF58w/build/asan/bin/kudu
/tmp/dist-test-taskRFF58w/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.60:44381
--webserver_interface=127.16.197.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.60.keytab
--principal=kudu/127.16.197.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.197.20:40249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.197.62:33109,127.16.197.61:46179,127.16.197.60:44381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 11:29:06.691020 9000 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } attempt: 1
I20260807 11:29:06.692288 9004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.691721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:44381 (local address 127.0.0.1:44856)
0807 11:29:06.691966 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:06.692101 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 11:29:06.693400 9004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.692985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:44381 (local address 127.0.0.1:44862)
0807 11:29:06.693170 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:06.693261 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 11:29:06.693624 9000 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.60:44381: Network error: Client connection negotiation failed: client connection to 127.16.197.60:44381: connect: Connection refused (error 111)
I20260807 11:29:06.758379 8922 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.61" port: 46179 } attempt: 5
I20260807 11:29:06.761866 9000 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } attempt: 2
I20260807 11:29:06.763059 9004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.762500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:44381 (local address 127.0.0.1:44874)
0807 11:29:06.762708 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:06.762831 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 11:29:06.764243 9004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.763826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:44381 (local address 127.0.0.1:44886)
0807 11:29:06.764023 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:06.764105 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260807 11:29:06.764855 9000 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.60:44381: Network error: Client connection negotiation failed: client connection to 127.16.197.60:44381: connect: Connection refused (error 111)
Aug 07 11:29:06 dist-test-slave-2w3w krb5kdc[8835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102145, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for kudu/127.16.197.61@KRBTEST.COM
I20260807 11:29:06.772938 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.759380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55886 (local address 127.16.197.61:46179)
0807 11:29:06.759578 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:06.759585 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:06.759606 (+ 21us) server_negotiation.cc:407] Connection header received
0807 11:29:06.759773 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:06.759778 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:06.759871 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:06.759986 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:06.760802 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:06.761626 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:06.763103 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:06.763785 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:06.763934 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:06.767412 (+ 3478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:06.767417 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:06.767426 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:06.767457 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:06.770218 (+ 2761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:06.770892 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:06.770898 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:06.770904 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:06.771088 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:06.771505 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:06.771510 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:06.771514 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:06.771802 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:06.771960 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:06.772372 (+ 412us) server_negotiation.cc:299] Negotiation successful
0807 11:29:06.772599 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 11:29:06.772959 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.759260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:46179 (local address 127.0.0.1:55886)
0807 11:29:06.759464 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:06.759487 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 11:29:06.759618 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:29:06.760022 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:29:06.760027 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:29:06.760461 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:29:06.760635 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:06.760644 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:06.761768 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:06.761774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:06.762915 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:06.762929 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:06.763952 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:06.763958 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:06.764173 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:06.764858 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:29:06.764885 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:29:06.767226 (+ 2341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:29:06.770364 (+ 3138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:06.770369 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:06.770374 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:06.770766 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:06.771207 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:06.771212 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:06.771216 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:06.771401 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:06.771932 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:29:06.771937 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:29:06.772103 (+ 166us) client_negotiation.cc:769] Sending connection context
0807 11:29:06.772271 (+ 168us) client_negotiation.cc:240] Negotiation successful
0807 11:29:06.772598 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 11:29:06.774412 8922 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } has no permanent_uuid. Determining permanent_uuid...
I20260807 11:29:06.775563 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.775195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:44381 (local address 127.0.0.1:44900)
0807 11:29:06.775358 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:06.775432 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 11:29:06.776499 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.776172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:44381 (local address 127.0.0.1:44902)
0807 11:29:06.776322 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:06.776406 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 11:29:06.776719 8922 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.60:44381: Network error: Client connection negotiation failed: client connection to 127.16.197.60:44381: connect: Connection refused (error 111)
I20260807 11:29:06.827939 8922 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } attempt: 1
I20260807 11:29:06.829213 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.828725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:44381 (local address 127.0.0.1:44904)
0807 11:29:06.828945 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:06.829047 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260807 11:29:06.830207 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.829878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:44381 (local address 127.0.0.1:44912)
0807 11:29:06.830041 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:06.830107 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 11:29:06.830415 8922 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.60:44381: Network error: Client connection negotiation failed: client connection to 127.16.197.60:44381: connect: Connection refused (error 111)
I20260807 11:29:06.856127 9000 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } attempt: 3
I20260807 11:29:06.857434 9004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.856967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:44381 (local address 127.0.0.1:44916)
0807 11:29:06.857171 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:06.857264 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 11:29:06.858577 9004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.858151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:44381 (local address 127.0.0.1:44926)
0807 11:29:06.858386 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:06.858467 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
W20260807 11:29:06.858776 9000 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.60:44381: Network error: Client connection negotiation failed: client connection to 127.16.197.60:44381: connect: Connection refused (error 111)
I20260807 11:29:06.898646 8922 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } attempt: 2
I20260807 11:29:06.899909 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.899395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:44381 (local address 127.0.0.1:44934)
0807 11:29:06.899646 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:06.899746 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260807 11:29:06.901072 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:06.900653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:44381 (local address 127.0.0.1:44942)
0807 11:29:06.900817 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:06.900903 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 11:29:06.901276 8922 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.60:44381: Network error: Client connection negotiation failed: client connection to 127.16.197.60:44381: connect: Connection refused (error 111)
I20260807 11:29:07.001979 9000 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } attempt: 4
I20260807 11:29:07.003207 9004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:07.002653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:44381 (local address 127.0.0.1:44950)
0807 11:29:07.002892 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:07.003040 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260807 11:29:07.003077 9009 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 11:29:07.003525 9009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 11:29:07.003648 9009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 11:29:07.003782 9009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 11:29:07.004354 9004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:07.003983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:44381 (local address 127.0.0.1:44952)
0807 11:29:07.004124 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:07.004210 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 11:29:07.004715 9000 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.60:44381: Network error: Client connection negotiation failed: client connection to 127.16.197.60:44381: connect: Connection refused (error 111)
I20260807 11:29:07.007469 8922 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } attempt: 3
I20260807 11:29:07.008457 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:07.008056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:44381 (local address 127.0.0.1:44966)
0807 11:29:07.008224 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:07.008306 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 11:29:07.009451 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:07.009073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:44381 (local address 127.0.0.1:44970)
0807 11:29:07.009285 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:07.009348 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.197.60:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260807 11:29:07.009654 8922 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.197.60:44381: Network error: Client connection negotiation failed: client connection to 127.16.197.60:44381: connect: Connection refused (error 111)
W20260807 11:29:07.012370 9009 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 11:29:07.012501 9009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 11:29:07.012616 9009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 11:29:07.012704 9009 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 11:29:07.012813 9009 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 11:29:07.023255 9009 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.197.20:40249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.16.197.62:33109,127.16.197.61:46179,127.16.197.60:44381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/krb5kdc/kudu_127.16.197.60.keytab
--principal=kudu/127.16.197.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.197.60:44381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.16.197.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 10:18:12 UTC on ccycloud.quasar-kutpcu.root.comops.site
build id 4572147
ASAN enabled
I20260807 11:29:07.024618 9009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 11:29:07.025997 9009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 11:29:07.037945 9014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:29:07.037837 9017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 11:29:07.038156 9015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 11:29:07.038594 9009 server_base.cc:1034] running on GCE node
I20260807 11:29:07.039502 9009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 11:29:07.040927 9009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 11:29:07.042182 9009 hybrid_clock.cc:648] HybridClock initialized: now 1786102147042107 us; error 50 us; skew 500 ppm
I20260807 11:29:07.042522 9009 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 11:29:07 dist-test-slave-2w3w krb5kdc[8835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102147, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 11:29:07.045996 9009 init.cc:377] Logged in from keytab as kudu/127.16.197.60@KRBTEST.COM (short username kudu)
I20260807 11:29:07.050238 9009 webserver.cc:460] Webserver started at http://127.16.197.60:44929/ using document root <none> and password file <none>
I20260807 11:29:07.051252 9009 fs_manager.cc:359] Metadata directory not provided
I20260807 11:29:07.051393 9009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 11:29:07.051834 9009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 11:29:07.055519 9009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-2/data/instance:
uuid: "5cd9eb5132804bc9b81cd7d54ea0fd9d"
format_stamp: "Formatted at 2026-08-07 11:29:07 on dist-test-slave-2w3w"
server_key: "c9bc3c8b2e2f1f7bd10392d2fe3ac3f2"
server_key_iv: "54bbd4f177e8a1aa8264093566caf6e2"
server_key_version: "encryptionkey@0"
I20260807 11:29:07.056325 9009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-2/wal/instance:
uuid: "5cd9eb5132804bc9b81cd7d54ea0fd9d"
format_stamp: "Formatted at 2026-08-07 11:29:07 on dist-test-slave-2w3w"
server_key: "c9bc3c8b2e2f1f7bd10392d2fe3ac3f2"
server_key_iv: "54bbd4f177e8a1aa8264093566caf6e2"
server_key_version: "encryptionkey@0"
I20260807 11:29:07.061872 9009 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 11:29:07.065371 9024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 11:29:07.066663 9009 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 11:29:07.066854 9009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-2/data,/tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-2/wal
uuid: "5cd9eb5132804bc9b81cd7d54ea0fd9d"
format_stamp: "Formatted at 2026-08-07 11:29:07 on dist-test-slave-2w3w"
server_key: "c9bc3c8b2e2f1f7bd10392d2fe3ac3f2"
server_key_iv: "54bbd4f177e8a1aa8264093566caf6e2"
server_key_version: "encryptionkey@0"
I20260807 11:29:07.067144 9009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 11:29:07.099226 9009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 11:29:07.103371 9009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 11:29:07.103669 9009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 11:29:07.122699 9009 rpc_server.cc:307] RPC server started. Bound to: 127.16.197.60:44381
I20260807 11:29:07.122717 9079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.197.60:44381 every 8 connection(s)
I20260807 11:29:07.124065 9009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-2/data/info.pb
I20260807 11:29:07.127755 9080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 11:29:07.128371 17172 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRFF58w/build/asan/bin/kudu as pid 9009
I20260807 11:29:07.128574 17172 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRFF58w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786101765888975-17172-0/minicluster-data/master-2/wal/instance
I20260807 11:29:07.129020 17172 external_mini_cluster.cc:1428] Setting key e39616a104053551fb29b8f8d410e9d8
I20260807 11:29:07.137669 9080 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33109 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 11:29:07 dist-test-slave-2w3w krb5kdc[8835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.60@KRBTEST.COM
Aug 07 11:29:07 dist-test-slave-2w3w krb5kdc[8835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102147, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.60@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:29:07.153719 9083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:07.130798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44974 (local address 127.16.197.60:44381)
0807 11:29:07.131665 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:07.131679 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:07.131728 (+ 49us) server_negotiation.cc:407] Connection header received
0807 11:29:07.134352 (+ 2624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:07.134392 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:07.134987 (+ 595us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:07.135443 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:07.136961 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.138926 (+ 1965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.140281 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.141427 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.141613 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:07.145948 (+ 4335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:07.145956 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:07.145966 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:07.146004 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:07.149477 (+ 3473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:07.150179 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:07.150194 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:07.150201 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:07.150416 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:07.150897 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:07.150901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:07.150903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:07.151570 (+ 667us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:07.151812 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:07.152308 (+ 496us) server_negotiation.cc:299] Negotiation successful
0807 11:29:07.152645 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":319,"threads_started":1}
I20260807 11:29:07.156013 9085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:07.140434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.62:33109 (local address 127.0.0.1:33410)
0807 11:29:07.141277 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:07.141315 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 11:29:07.141515 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:29:07.142009 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:29:07.142018 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:29:07.142868 (+ 850us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:29:07.143270 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:07.143291 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.144281 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.144287 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:07.145395 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:07.145408 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.146167 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.146174 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:07.146372 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:07.147126 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:29:07.147146 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:29:07.149504 (+ 2358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:29:07.153176 (+ 3672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:07.153189 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:07.153196 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:07.153596 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:07.154056 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:07.154061 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:07.154064 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:07.154198 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:07.154729 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:29:07.154737 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:29:07.155006 (+ 269us) client_negotiation.cc:769] Sending connection context
0807 11:29:07.155232 (+ 226us) client_negotiation.cc:240] Negotiation successful
0807 11:29:07.155551 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":526,"threads_started":1}
I20260807 11:29:07.155979 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:07.140407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33410 (local address 127.16.197.62:33109)
0807 11:29:07.140965 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:07.140973 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:07.141430 (+ 457us) server_negotiation.cc:407] Connection header received
0807 11:29:07.141680 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:07.141685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:07.141786 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:07.141886 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:07.143446 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.144122 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.145551 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.145966 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.146107 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:07.149704 (+ 3597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:07.149709 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:07.149712 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:07.149753 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:07.153032 (+ 3279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:07.153735 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:07.153741 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:07.153745 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:07.153945 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:07.154308 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:07.154315 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:07.154321 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:07.154588 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:07.154755 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:07.155341 (+ 586us) server_negotiation.cc:299] Negotiation successful
0807 11:29:07.155573 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":299,"threads_started":1}
I20260807 11:29:07.157655 9080 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.61" port: 46179 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 11:29:07 dist-test-slave-2w3w krb5kdc[8835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102147, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.60@KRBTEST.COM for kudu/127.16.197.61@KRBTEST.COM
Aug 07 11:29:07 dist-test-slave-2w3w krb5kdc[8835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.62@KRBTEST.COM
I20260807 11:29:07.170630 9085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:07.158458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.61:46179 (local address 127.0.0.1:55902)
0807 11:29:07.158638 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:07.158657 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 11:29:07.158747 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:29:07.159095 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:29:07.159100 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:29:07.159475 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:29:07.159605 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:07.159612 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.160565 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.160571 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:07.161429 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:07.161440 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.162164 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.162170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:07.162361 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:07.162881 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:29:07.162905 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:29:07.164974 (+ 2069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:29:07.168223 (+ 3249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:07.168229 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:07.168233 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:07.168587 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:07.168974 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:07.168979 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:07.168983 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:07.169147 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:07.169655 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:29:07.169659 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:29:07.169807 (+ 148us) client_negotiation.cc:769] Sending connection context
0807 11:29:07.169951 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 11:29:07.170255 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 11:29:07.170661 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:07.158553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55902 (local address 127.16.197.61:46179)
0807 11:29:07.158743 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:07.158749 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:07.158769 (+ 20us) server_negotiation.cc:407] Connection header received
0807 11:29:07.158861 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:07.158866 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:07.158940 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:07.159076 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:07.159751 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.160432 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.161587 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.162012 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.162148 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:07.165173 (+ 3025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:07.165178 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:07.165181 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:07.165212 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:07.168089 (+ 2877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:07.168700 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:07.168706 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:07.168710 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:07.168850 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:07.169256 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:07.169261 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:07.169264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:07.169526 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:07.169687 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:07.170045 (+ 358us) server_negotiation.cc:299] Negotiation successful
0807 11:29:07.170289 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 11:29:07.171226 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:07.157095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33414 (local address 127.16.197.62:33109)
0807 11:29:07.157236 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:07.157243 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:07.157263 (+ 20us) server_negotiation.cc:407] Connection header received
0807 11:29:07.157306 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:07.157309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:07.157368 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:07.157479 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:07.158935 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.159653 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.160845 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.161246 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.161399 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:07.165751 (+ 4352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:07.165756 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:07.165759 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:07.165788 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:07.168288 (+ 2500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:07.168957 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:07.168961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:07.168963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:07.169097 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:07.169483 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:07.169488 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:07.169492 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:07.169763 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:07.170065 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:07.170668 (+ 603us) server_negotiation.cc:299] Negotiation successful
0807 11:29:07.170911 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 11:29:07.172081 9080 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 11:29:07 dist-test-slave-2w3w krb5kdc[8835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102147, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.60@KRBTEST.COM for kudu/127.16.197.60@KRBTEST.COM
Aug 07 11:29:07 dist-test-slave-2w3w krb5kdc[8835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.197.61@KRBTEST.COM
I20260807 11:29:07.184687 9085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:07.172859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:44381 (local address 127.0.0.1:44984)
0807 11:29:07.173039 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:07.173057 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 11:29:07.173161 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:29:07.173457 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:29:07.173461 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:29:07.173821 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:29:07.173943 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:07.173951 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.174882 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.174888 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:07.175800 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:07.175812 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.176492 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.176499 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:07.176712 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:07.177262 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:29:07.177278 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:29:07.179216 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:29:07.182256 (+ 3040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:07.182263 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:07.182268 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:07.182603 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:07.182951 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:07.182956 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:07.183001 (+ 45us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:07.183147 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:07.183665 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:29:07.183670 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:29:07.183808 (+ 138us) client_negotiation.cc:769] Sending connection context
0807 11:29:07.183964 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 11:29:07.184278 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260807 11:29:07.184741 9083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:07.172845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44984 (local address 127.16.197.60:44381)
0807 11:29:07.173035 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:07.173041 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:07.173106 (+ 65us) server_negotiation.cc:407] Connection header received
0807 11:29:07.173277 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:07.173284 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:07.173343 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:07.173442 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:07.174084 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.174743 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.175941 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.176334 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.176477 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:07.179391 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:07.179395 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:07.179399 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:07.179425 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:07.182124 (+ 2699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:07.182717 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:07.182724 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:07.182727 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:07.182837 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:07.183252 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:07.183260 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:07.183263 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:07.183551 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:07.183700 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:07.184058 (+ 358us) server_negotiation.cc:299] Negotiation successful
0807 11:29:07.184442 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":156,"server-negotiator.queue_time_us":75}
I20260807 11:29:07.186872 8922 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } attempt: 4
I20260807 11:29:07.188903 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:07.173547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55906 (local address 127.16.197.61:46179)
0807 11:29:07.173697 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:07.173702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:07.173790 (+ 88us) server_negotiation.cc:407] Connection header received
0807 11:29:07.173966 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:07.173971 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:07.174056 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:07.174149 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:07.175376 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.176075 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.177353 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.177779 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.177911 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:07.182510 (+ 4599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:07.182515 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:07.182518 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:07.182549 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:07.185541 (+ 2992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:07.186346 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:07.186350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:07.186352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:07.186488 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:07.186908 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:07.186913 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:07.186917 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:07.187302 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:07.187611 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:07.188322 (+ 711us) server_negotiation.cc:299] Negotiation successful
0807 11:29:07.188561 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 11:29:07.191314 9080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cd9eb5132804bc9b81cd7d54ea0fd9d: Bootstrap starting.
I20260807 11:29:07.194144 17172 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
Aug 07 11:29:07 dist-test-slave-2w3w krb5kdc[8835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102145, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.62@KRBTEST.COM for kudu/127.16.197.60@KRBTEST.COM
I20260807 11:29:07.195622 9080 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5cd9eb5132804bc9b81cd7d54ea0fd9d: Neither blocks nor log segments found. Creating new log.
I20260807 11:29:07.197000 9080 log.cc:824] T 00000000000000000000000000000000 P 5cd9eb5132804bc9b81cd7d54ea0fd9d: Log is configured to *not* fsync() on all Append() calls
I20260807 11:29:07.197404 8895 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "5bb10dc3be6f49448bb6cced3affc0fa"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33414
I20260807 11:29:07.201944 9080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cd9eb5132804bc9b81cd7d54ea0fd9d: No bootstrap required, opened a new log
I20260807 11:29:07.203840 9083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:07.187945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44988 (local address 127.16.197.60:44381)
0807 11:29:07.188141 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:07.188147 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:07.188163 (+ 16us) server_negotiation.cc:407] Connection header received
0807 11:29:07.188214 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:07.188224 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:07.188301 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:07.189358 (+ 1057us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:07.189492 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.190530 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.192317 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.192895 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.193030 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:07.196465 (+ 3435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:07.196471 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:07.196477 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:07.196512 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:07.200457 (+ 3945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:07.201408 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:07.201414 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:07.201417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:07.201589 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:07.202067 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:07.202086 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:07.202091 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:07.202552 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:07.202714 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:07.203217 (+ 503us) server_negotiation.cc:299] Negotiation successful
0807 11:29:07.203444 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 11:29:07.203830 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:07.187572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:44381 (local address 127.0.0.1:44988)
0807 11:29:07.187743 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:07.187759 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 11:29:07.187882 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:29:07.188488 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:29:07.188494 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:29:07.189018 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:29:07.189214 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:07.189226 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.190691 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.190700 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:07.192182 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:07.192195 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.193055 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.193061 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:07.193290 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:07.194017 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:29:07.194039 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:29:07.196271 (+ 2232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:29:07.200633 (+ 4362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:07.200647 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:07.200650 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:07.201262 (+ 612us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:07.201746 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:07.201752 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:07.201758 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:07.201944 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:07.202688 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:29:07.202694 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:29:07.202870 (+ 176us) client_negotiation.cc:769] Sending connection context
0807 11:29:07.203105 (+ 235us) client_negotiation.cc:240] Negotiation successful
0807 11:29:07.203425 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 11:29:07.208505 8922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bb10dc3be6f49448bb6cced3affc0fa: Bootstrap starting.
I20260807 11:29:07.211663 8922 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5bb10dc3be6f49448bb6cced3affc0fa: Neither blocks nor log segments found. Creating new log.
I20260807 11:29:07.212894 8922 log.cc:824] T 00000000000000000000000000000000 P 5bb10dc3be6f49448bb6cced3affc0fa: Log is configured to *not* fsync() on all Append() calls
I20260807 11:29:07.212672 9080 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5cd9eb5132804bc9b81cd7d54ea0fd9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bb10dc3be6f49448bb6cced3affc0fa" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33109 } } peers { permanent_uuid: "e0178f812b5947b0824f6830b68f5925" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 46179 } } peers { permanent_uuid: "5cd9eb5132804bc9b81cd7d54ea0fd9d" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } }
I20260807 11:29:07.213104 9080 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5cd9eb5132804bc9b81cd7d54ea0fd9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd9eb5132804bc9b81cd7d54ea0fd9d, State: Initialized, Role: FOLLOWER
I20260807 11:29:07.213590 9080 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5cd9eb5132804bc9b81cd7d54ea0fd9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bb10dc3be6f49448bb6cced3affc0fa" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33109 } } peers { permanent_uuid: "e0178f812b5947b0824f6830b68f5925" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 46179 } } peers { permanent_uuid: "5cd9eb5132804bc9b81cd7d54ea0fd9d" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } }
I20260807 11:29:07.215021 9088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cd9eb5132804bc9b81cd7d54ea0fd9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bb10dc3be6f49448bb6cced3affc0fa" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33109 } } peers { permanent_uuid: "e0178f812b5947b0824f6830b68f5925" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 46179 } } peers { permanent_uuid: "5cd9eb5132804bc9b81cd7d54ea0fd9d" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } } }
I20260807 11:29:07.215843 9088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cd9eb5132804bc9b81cd7d54ea0fd9d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 11:29:07.216189 9080 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5cd9eb5132804bc9b81cd7d54ea0fd9d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:29:07.216944 8922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bb10dc3be6f49448bb6cced3affc0fa: No bootstrap required, opened a new log
I20260807 11:29:07.223361 8922 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5bb10dc3be6f49448bb6cced3affc0fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bb10dc3be6f49448bb6cced3affc0fa" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33109 } } peers { permanent_uuid: "e0178f812b5947b0824f6830b68f5925" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 46179 } } peers { permanent_uuid: "5cd9eb5132804bc9b81cd7d54ea0fd9d" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } }
I20260807 11:29:07.223781 8922 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5bb10dc3be6f49448bb6cced3affc0fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bb10dc3be6f49448bb6cced3affc0fa, State: Initialized, Role: FOLLOWER
I20260807 11:29:07.224299 8922 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5bb10dc3be6f49448bb6cced3affc0fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bb10dc3be6f49448bb6cced3affc0fa" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33109 } } peers { permanent_uuid: "e0178f812b5947b0824f6830b68f5925" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 46179 } } peers { permanent_uuid: "5cd9eb5132804bc9b81cd7d54ea0fd9d" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } }
I20260807 11:29:07.225574 9100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bb10dc3be6f49448bb6cced3affc0fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bb10dc3be6f49448bb6cced3affc0fa" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33109 } } peers { permanent_uuid: "e0178f812b5947b0824f6830b68f5925" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 46179 } } peers { permanent_uuid: "5cd9eb5132804bc9b81cd7d54ea0fd9d" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } } }
I20260807 11:29:07.226259 9100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bb10dc3be6f49448bb6cced3affc0fa [sys.catalog]: This master's current role is: FOLLOWER
I20260807 11:29:07.226785 8922 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5bb10dc3be6f49448bb6cced3affc0fa [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:29:07.226883 8895 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5bb10dc3be6f49448bb6cced3affc0fa [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 11:29:07.227092 8895 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5bb10dc3be6f49448bb6cced3affc0fa [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:29:07.231334 8895 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5bb10dc3be6f49448bb6cced3affc0fa [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bb10dc3be6f49448bb6cced3affc0fa" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33109 } } peers { permanent_uuid: "e0178f812b5947b0824f6830b68f5925" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 46179 } } peers { permanent_uuid: "5cd9eb5132804bc9b81cd7d54ea0fd9d" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } }
I20260807 11:29:07.233435 8895 leader_election.cc:288] T 00000000000000000000000000000000 P 5bb10dc3be6f49448bb6cced3affc0fa [CANDIDATE]: Term 1 election: Requested vote from peers e0178f812b5947b0824f6830b68f5925 (127.16.197.61:46179), 5cd9eb5132804bc9b81cd7d54ea0fd9d (127.16.197.60:44381)
I20260807 11:29:07.235212 9053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5bb10dc3be6f49448bb6cced3affc0fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5cd9eb5132804bc9b81cd7d54ea0fd9d"
I20260807 11:29:07.235786 9053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5cd9eb5132804bc9b81cd7d54ea0fd9d [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:29:07.240803 9053 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5cd9eb5132804bc9b81cd7d54ea0fd9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bb10dc3be6f49448bb6cced3affc0fa in term 1.
I20260807 11:29:07.241613 8869 leader_election.cc:302] T 00000000000000000000000000000000 P 5bb10dc3be6f49448bb6cced3affc0fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5bb10dc3be6f49448bb6cced3affc0fa, 5cd9eb5132804bc9b81cd7d54ea0fd9d; no voters:
I20260807 11:29:07.242274 9100 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5bb10dc3be6f49448bb6cced3affc0fa [term 1 FOLLOWER]: Leader election won for term 1
I20260807 11:29:07.248258 9100 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5bb10dc3be6f49448bb6cced3affc0fa [term 1 LEADER]: Becoming Leader. State: Replica: 5bb10dc3be6f49448bb6cced3affc0fa, State: Running, Role: LEADER
I20260807 11:29:07.249089 9100 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5bb10dc3be6f49448bb6cced3affc0fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bb10dc3be6f49448bb6cced3affc0fa" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33109 } } peers { permanent_uuid: "e0178f812b5947b0824f6830b68f5925" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 46179 } } peers { permanent_uuid: "5cd9eb5132804bc9b81cd7d54ea0fd9d" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } }
I20260807 11:29:07.225484 9009 master_runner.cc:186] Error getting master registration for 127.16.197.62:33109: OK, instance_id { permanent_uuid: "5bb10dc3be6f49448bb6cced3affc0fa" instance_seqno: 1786102146048852 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 11:29:07.260074 9104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bb10dc3be6f49448bb6cced3affc0fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bb10dc3be6f49448bb6cced3affc0fa. Latest consensus state: current_term: 1 leader_uuid: "5bb10dc3be6f49448bb6cced3affc0fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bb10dc3be6f49448bb6cced3affc0fa" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33109 } } peers { permanent_uuid: "e0178f812b5947b0824f6830b68f5925" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 46179 } } peers { permanent_uuid: "5cd9eb5132804bc9b81cd7d54ea0fd9d" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } } }
I20260807 11:29:07.260556 9104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bb10dc3be6f49448bb6cced3affc0fa [sys.catalog]: This master's current role is: LEADER
I20260807 11:29:07.260972 9009 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 11:29:07.261909 9114 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260807 11:29:07.262928 9099 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5cd9eb5132804bc9b81cd7d54ea0fd9d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 11:29:07.263092 9099 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260807 11:29:07.288192 8872 tablet.cc:2369] T 00000000000000000000000000000000 P 5bb10dc3be6f49448bb6cced3affc0fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:29:07.298292 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:07.262003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33422 (local address 127.16.197.62:33109)
0807 11:29:07.262196 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:07.262202 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:07.272463 (+ 10261us) server_negotiation.cc:407] Connection header received
0807 11:29:07.272952 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:07.272958 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:07.273043 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:07.273122 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:07.275617 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.276784 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.280838 (+ 4054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.281562 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.281677 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:07.289270 (+ 7593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:07.289279 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:07.289284 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:07.289352 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:07.293242 (+ 3890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:07.294817 (+ 1575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:07.294823 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:07.294827 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:07.295049 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:07.295967 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:07.295972 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:07.295976 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:07.296341 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:07.296632 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:07.297641 (+ 1009us) server_negotiation.cc:299] Negotiation successful
0807 11:29:07.297881 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 11:29:07.303901 9000 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } attempt: 5
I20260807 11:29:07.317724 9114 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 11:29:07.321293 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:07.262083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55912 (local address 127.16.197.61:46179)
0807 11:29:07.262272 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:07.262278 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:07.280726 (+ 18448us) server_negotiation.cc:407] Connection header received
0807 11:29:07.281315 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:07.281323 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:07.281446 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:07.281538 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:07.285570 (+ 4032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.287085 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.293607 (+ 6522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.294477 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.294609 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:07.310255 (+ 15646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:07.310265 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:07.310271 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:07.310326 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:07.314768 (+ 4442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:07.317857 (+ 3089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:07.317863 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:07.317865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:07.318062 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:07.318645 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:07.318653 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:07.318659 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:07.319221 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:07.319555 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:07.320632 (+ 1077us) server_negotiation.cc:299] Negotiation successful
0807 11:29:07.320899 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 11:29:07 dist-test-slave-2w3w krb5kdc[8835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786102146, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.197.61@KRBTEST.COM for kudu/127.16.197.60@KRBTEST.COM
I20260807 11:29:07.334131 9083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:07.265091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45000 (local address 127.16.197.60:44381)
0807 11:29:07.265261 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:07.265267 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:07.285082 (+ 19815us) server_negotiation.cc:407] Connection header received
0807 11:29:07.289508 (+ 4426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:07.289518 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:07.289626 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:07.289717 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:07.295872 (+ 6155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.297295 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.304476 (+ 7181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.305239 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.305355 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:07.323859 (+ 18504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:07.323866 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:07.323870 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:07.323914 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:07.327820 (+ 3906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:07.331279 (+ 3459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:07.331285 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:07.331288 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:07.331462 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:07.331951 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:07.331955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:07.331958 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:07.332388 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:07.332678 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:07.333316 (+ 638us) server_negotiation.cc:299] Negotiation successful
0807 11:29:07.333604 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58,"spinlock_wait_cycles":30464}
I20260807 11:29:07.345065 9004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:07.304594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.197.60:44381 (local address 127.0.0.1:45014)
0807 11:29:07.305026 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0807 11:29:07.305046 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 11:29:07.305160 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 11:29:07.321188 (+ 16028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 11:29:07.321193 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 11:29:07.321702 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0807 11:29:07.321909 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:07.321932 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.325313 (+ 3381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.325319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:07.326878 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 11:29:07.326894 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.328112 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.328122 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 11:29:07.328352 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:07.328918 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 11:29:07.328953 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 11:29:07.331822 (+ 2869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 11:29:07.342115 (+ 10293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:07.342122 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:07.342126 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:07.342695 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:07.343234 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 11:29:07.343239 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 11:29:07.343243 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 11:29:07.343402 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 11:29:07.344041 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 11:29:07.344046 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 11:29:07.344223 (+ 177us) client_negotiation.cc:769] Sending connection context
0807 11:29:07.344372 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 11:29:07.344655 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283}
I20260807 11:29:07.346544 9122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:07.305364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45014 (local address 127.16.197.60:44381)
0807 11:29:07.320744 (+ 15380us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:07.320751 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:07.320774 (+ 23us) server_negotiation.cc:407] Connection header received
0807 11:29:07.320835 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:07.320840 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:07.320918 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:07.322333 (+ 1415us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:07.322620 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.325134 (+ 2514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.327185 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.327912 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.328765 (+ 853us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:07.332176 (+ 3411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 11:29:07.332183 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 11:29:07.332187 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 11:29:07.332217 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 11:29:07.341948 (+ 9731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:07.342874 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:07.342879 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:07.342883 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:07.343104 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 11:29:07.343549 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 11:29:07.343554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 11:29:07.343558 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 11:29:07.343888 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 11:29:07.345692 (+ 1804us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:07.345823 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 11:29:07.346040 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15237,"thread_start_us":15638,"threads_started":1}
I20260807 11:29:07.354467 9000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0178f812b5947b0824f6830b68f5925: Bootstrap starting.
W20260807 11:29:07.355960 9032 tablet.cc:2369] T 00000000000000000000000000000000 P 5cd9eb5132804bc9b81cd7d54ea0fd9d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 11:29:07.381156 9000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0178f812b5947b0824f6830b68f5925: Neither blocks nor log segments found. Creating new log.
I20260807 11:29:07.383260 9000 log.cc:824] T 00000000000000000000000000000000 P e0178f812b5947b0824f6830b68f5925: Log is configured to *not* fsync() on all Append() calls
I20260807 11:29:07.364344 8851 master_runner.cc:186] Error getting master registration for 127.16.197.61:46179: OK, instance_id { permanent_uuid: "e0178f812b5947b0824f6830b68f5925" instance_seqno: 1786102146571782 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 11:29:07.389866 8851 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 11:29:07.390924 9053 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5cd9eb5132804bc9b81cd7d54ea0fd9d [term 1 FOLLOWER]: Refusing update from remote peer 5bb10dc3be6f49448bb6cced3affc0fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:29:07.392030 9104 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5bb10dc3be6f49448bb6cced3affc0fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd9eb5132804bc9b81cd7d54ea0fd9d" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:29:07.403239 9000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0178f812b5947b0824f6830b68f5925: No bootstrap required, opened a new log
I20260807 11:29:07.420011 9088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cd9eb5132804bc9b81cd7d54ea0fd9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bb10dc3be6f49448bb6cced3affc0fa. Latest consensus state: current_term: 1 leader_uuid: "5bb10dc3be6f49448bb6cced3affc0fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bb10dc3be6f49448bb6cced3affc0fa" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33109 } } peers { permanent_uuid: "e0178f812b5947b0824f6830b68f5925" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 46179 } } peers { permanent_uuid: "5cd9eb5132804bc9b81cd7d54ea0fd9d" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } } }
I20260807 11:29:07.420434 9088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cd9eb5132804bc9b81cd7d54ea0fd9d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 11:29:07.423511 9125 mvcc.cc:205] Tried to move back new op lower bound from 7315874395702104064 to 7315874395141083136. Current Snapshot: MvccSnapshot[applied={T|T < 7315874395702104064}]
I20260807 11:29:07.424273 9129 mvcc.cc:205] Tried to move back new op lower bound from 7315874395702104064 to 7315874395141083136. Current Snapshot: MvccSnapshot[applied={T|T < 7315874395702104064}]
I20260807 11:29:07.436108 9100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bb10dc3be6f49448bb6cced3affc0fa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5bb10dc3be6f49448bb6cced3affc0fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bb10dc3be6f49448bb6cced3affc0fa" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33109 } } peers { permanent_uuid: "e0178f812b5947b0824f6830b68f5925" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 46179 } } peers { permanent_uuid: "5cd9eb5132804bc9b81cd7d54ea0fd9d" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } } }
I20260807 11:29:07.436491 9100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bb10dc3be6f49448bb6cced3affc0fa [sys.catalog]: This master's current role is: LEADER
I20260807 11:29:07.438038 9000 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0178f812b5947b0824f6830b68f5925 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bb10dc3be6f49448bb6cced3affc0fa" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33109 } } peers { permanent_uuid: "e0178f812b5947b0824f6830b68f5925" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 46179 } } peers { permanent_uuid: "5cd9eb5132804bc9b81cd7d54ea0fd9d" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } }
I20260807 11:29:07.438530 9000 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0178f812b5947b0824f6830b68f5925 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0178f812b5947b0824f6830b68f5925, State: Initialized, Role: FOLLOWER
I20260807 11:29:07.439143 9000 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0178f812b5947b0824f6830b68f5925 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bb10dc3be6f49448bb6cced3affc0fa" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33109 } } peers { permanent_uuid: "e0178f812b5947b0824f6830b68f5925" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 46179 } } peers { permanent_uuid: "5cd9eb5132804bc9b81cd7d54ea0fd9d" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } }
I20260807 11:29:07.440575 9088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cd9eb5132804bc9b81cd7d54ea0fd9d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5bb10dc3be6f49448bb6cced3affc0fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bb10dc3be6f49448bb6cced3affc0fa" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33109 } } peers { permanent_uuid: "e0178f812b5947b0824f6830b68f5925" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 46179 } } peers { permanent_uuid: "5cd9eb5132804bc9b81cd7d54ea0fd9d" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } } }
I20260807 11:29:07.440850 9088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cd9eb5132804bc9b81cd7d54ea0fd9d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 11:29:07.443128 9114 catalog_manager.cc:1347] Generated new cluster ID: ff3a8ad7f6224300b4491c72fc8016f2
I20260807 11:29:07.443243 9114 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 11:29:07.445549 9133 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0178f812b5947b0824f6830b68f5925 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bb10dc3be6f49448bb6cced3affc0fa" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33109 } } peers { permanent_uuid: "e0178f812b5947b0824f6830b68f5925" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 46179 } } peers { permanent_uuid: "5cd9eb5132804bc9b81cd7d54ea0fd9d" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } } }
I20260807 11:29:07.446046 9133 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0178f812b5947b0824f6830b68f5925 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 11:29:07.448697 9000 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0178f812b5947b0824f6830b68f5925 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 11:29:07.450264 8971 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0178f812b5947b0824f6830b68f5925 [term 0 FOLLOWER]: Advancing to term 1
I20260807 11:29:07.454141 8971 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e0178f812b5947b0824f6830b68f5925 [term 1 FOLLOWER]: Refusing update from remote peer 5bb10dc3be6f49448bb6cced3affc0fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 11:29:07.455492 8972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5bb10dc3be6f49448bb6cced3affc0fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e0178f812b5947b0824f6830b68f5925"
I20260807 11:29:07.459101 8972 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e0178f812b5947b0824f6830b68f5925 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bb10dc3be6f49448bb6cced3affc0fa in term 1.
I20260807 11:29:07.461053 9104 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5bb10dc3be6f49448bb6cced3affc0fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0178f812b5947b0824f6830b68f5925" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 46179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 11:29:07.507388 9114 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 11:29:07.511166 9114 catalog_manager.cc:1504] Loading token signing keys...
I20260807 11:29:07.517190 9141 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0178f812b5947b0824f6830b68f5925 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5bb10dc3be6f49448bb6cced3affc0fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bb10dc3be6f49448bb6cced3affc0fa" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33109 } } peers { permanent_uuid: "e0178f812b5947b0824f6830b68f5925" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 46179 } } peers { permanent_uuid: "5cd9eb5132804bc9b81cd7d54ea0fd9d" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } } }
I20260807 11:29:07.517602 9141 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0178f812b5947b0824f6830b68f5925 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 11:29:07.517987 9133 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0178f812b5947b0824f6830b68f5925 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bb10dc3be6f49448bb6cced3affc0fa. Latest consensus state: current_term: 1 leader_uuid: "5bb10dc3be6f49448bb6cced3affc0fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bb10dc3be6f49448bb6cced3affc0fa" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33109 } } peers { permanent_uuid: "e0178f812b5947b0824f6830b68f5925" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 46179 } } peers { permanent_uuid: "5cd9eb5132804bc9b81cd7d54ea0fd9d" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } } }
I20260807 11:29:07.518253 9133 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0178f812b5947b0824f6830b68f5925 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 11:29:07.519439 9104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bb10dc3be6f49448bb6cced3affc0fa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5bb10dc3be6f49448bb6cced3affc0fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bb10dc3be6f49448bb6cced3affc0fa" member_type: VOTER last_known_addr { host: "127.16.197.62" port: 33109 } } peers { permanent_uuid: "e0178f812b5947b0824f6830b68f5925" member_type: VOTER last_known_addr { host: "127.16.197.61" port: 46179 } } peers { permanent_uuid: "5cd9eb5132804bc9b81cd7d54ea0fd9d" member_type: VOTER last_known_addr { host: "127.16.197.60" port: 44381 } } }
I20260807 11:29:07.519701 9104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bb10dc3be6f49448bb6cced3affc0fa [sys.catalog]: This master's current role is: LEADER
I20260807 11:29:07.539896 9114 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5bb10dc3be6f49448bb6cced3affc0fa: Generated new TSK 0
I20260807 11:29:07.541060 9114 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 11:29:07.586143 9152 catalog_manager.cc:1259] Loaded cluster ID: ff3a8ad7f6224300b4491c72fc8016f2
I20260807 11:29:07.586357 9152 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e0178f812b5947b0824f6830b68f5925: loading cluster ID for follower catalog manager: success
I20260807 11:29:07.590553 9152 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e0178f812b5947b0824f6830b68f5925: acquiring CA information for follower catalog manager: success
I20260807 11:29:07.609030 9152 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e0178f812b5947b0824f6830b68f5925: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 11:29:07.719008 9084 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:29:07.699130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33434 (local address 127.16.197.62:33109)
0807 11:29:07.699374 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:07.699380 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:07.714838 (+ 15458us) server_negotiation.cc:407] Connection header received
0807 11:29:07.717294 (+ 2456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:07.717303 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:07.717409 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:07.717498 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:07.718700 (+ 1202us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33434: BlockingRecv error: recv got EOF from 127.0.0.1:33434 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260807 11:29:07.721833 9122 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:29:07.705485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45020 (local address 127.16.197.60:44381)
0807 11:29:07.705697 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:07.705704 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:07.716978 (+ 11274us) server_negotiation.cc:407] Connection header received
0807 11:29:07.719832 (+ 2854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:07.719840 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:07.719957 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:07.720038 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:07.721562 (+ 1524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45020: BlockingRecv error: recv got EOF from 127.0.0.1:45020 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260807 11:29:07.725428 9006 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:29:07.711625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55926 (local address 127.16.197.61:46179)
0807 11:29:07.711837 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:07.711844 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:07.723448 (+ 11604us) server_negotiation.cc:407] Connection header received
0807 11:29:07.723981 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:07.723989 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:07.724103 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:07.724183 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:07.725186 (+ 1003us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55926: BlockingRecv error: recv got EOF from 127.0.0.1:55926 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 11:29:07.773073 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:07.757498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55936 (local address 127.16.197.61:46179)
0807 11:29:07.757702 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:07.757709 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:07.764626 (+ 6917us) server_negotiation.cc:407] Connection header received
0807 11:29:07.765231 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:07.765237 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:07.765469 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:07.765616 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:29:07.766429 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.767626 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.769730 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.770423 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.770533 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:07.771371 (+ 838us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:29:07.772136 (+ 765us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:29:07.772263 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:07.772576 (+ 313us) server_negotiation.cc:299] Negotiation successful
0807 11:29:07.772685 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260807 11:29:07.817516 9122 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:29:07.807195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45036 (local address 127.16.197.60:44381)
0807 11:29:07.807535 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:07.807542 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:07.815089 (+ 7547us) server_negotiation.cc:407] Connection header received
0807 11:29:07.815705 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:07.815712 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:07.815875 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:07.815983 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:07.817213 (+ 1230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45036: BlockingRecv error: recv got EOF from 127.0.0.1:45036 (error 108)
Metrics: {"server-negotiator.queue_time_us":173}
I20260807 11:29:07.863979 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:07.849613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55938 (local address 127.16.197.61:46179)
0807 11:29:07.849839 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:07.849846 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:07.856802 (+ 6956us) server_negotiation.cc:407] Connection header received
0807 11:29:07.857352 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:07.857360 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:07.857487 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:07.857567 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:29:07.858278 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.859476 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.861554 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.862204 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.862318 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:07.863045 (+ 727us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:29:07.863208 (+ 163us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:29:07.863285 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:07.863542 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 11:29:07.863638 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260807 11:29:07.908670 9122 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:29:07.898537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45038 (local address 127.16.197.60:44381)
0807 11:29:07.898837 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:07.898845 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:07.906390 (+ 7545us) server_negotiation.cc:407] Connection header received
0807 11:29:07.906956 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:07.906965 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:07.907091 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:07.907178 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:07.908412 (+ 1234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45038: BlockingRecv error: recv got EOF from 127.0.0.1:45038 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 11:29:07.961163 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:07.946808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55952 (local address 127.16.197.61:46179)
0807 11:29:07.947066 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:07.947073 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:07.954087 (+ 7014us) server_negotiation.cc:407] Connection header received
0807 11:29:07.954599 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:07.954606 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:07.954695 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:07.954799 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:29:07.955514 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.956684 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.958768 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:07.959490 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:07.959593 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:07.960231 (+ 638us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:29:07.960397 (+ 166us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:29:07.960459 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:07.960685 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 11:29:07.960798 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
W20260807 11:29:07.996263 9122 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:29:07.993791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45052 (local address 127.16.197.60:44381)
0807 11:29:07.994019 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:07.994030 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:07.994639 (+ 609us) server_negotiation.cc:407] Connection header received
0807 11:29:07.994892 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:07.994898 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:07.995043 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:07.995161 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:07.995993 (+ 832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45052: BlockingRecv error: recv got EOF from 127.0.0.1:45052 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 11:29:08.010110 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:08.003473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55966 (local address 127.16.197.61:46179)
0807 11:29:08.003718 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:08.003728 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:08.004299 (+ 571us) server_negotiation.cc:407] Connection header received
0807 11:29:08.004425 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:08.004431 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:08.004551 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:08.004672 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:29:08.005125 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:08.006328 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:08.008087 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:08.008796 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:08.008941 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:08.009323 (+ 382us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:29:08.009467 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:29:08.009585 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:08.009703 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 11:29:08.009804 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260807 11:29:08.019407 9122 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:29:08.016991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45058 (local address 127.16.197.60:44381)
0807 11:29:08.017312 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:08.017319 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:08.017561 (+ 242us) server_negotiation.cc:407] Connection header received
0807 11:29:08.017822 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:08.017829 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:08.017920 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:08.018030 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:08.019109 (+ 1079us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45058: BlockingRecv error: recv got EOF from 127.0.0.1:45058 (error 108)
Metrics: {"server-negotiator.queue_time_us":172}
I20260807 11:29:08.036875 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:08.030682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55974 (local address 127.16.197.61:46179)
0807 11:29:08.030902 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:08.030909 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:08.031266 (+ 357us) server_negotiation.cc:407] Connection header received
0807 11:29:08.031422 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:08.031428 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:08.031519 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:08.031635 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:29:08.032038 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:08.033164 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:08.034476 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:08.035349 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:08.035484 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:08.035896 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:29:08.036068 (+ 172us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:29:08.036172 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:08.036414 (+ 242us) server_negotiation.cc:299] Negotiation successful
0807 11:29:08.036512 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260807 11:29:08.046151 9122 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:29:08.044133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45070 (local address 127.16.197.60:44381)
0807 11:29:08.044341 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:08.044347 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:08.044664 (+ 317us) server_negotiation.cc:407] Connection header received
0807 11:29:08.044853 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:08.044857 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:08.044937 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:08.045019 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:08.045950 (+ 931us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45070: BlockingRecv error: recv got EOF from 127.0.0.1:45070 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 11:29:08.075364 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:08.069508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55976 (local address 127.16.197.61:46179)
0807 11:29:08.069713 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:08.069720 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:08.070089 (+ 369us) server_negotiation.cc:407] Connection header received
0807 11:29:08.070357 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:08.070361 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:08.070432 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:08.070533 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:29:08.071039 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:08.071972 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:08.073421 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:08.073869 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:08.074023 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:08.074504 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:29:08.074612 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:29:08.074773 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:08.074946 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 11:29:08.075075 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260807 11:29:08.084924 9122 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:29:08.082740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45086 (local address 127.16.197.60:44381)
0807 11:29:08.083014 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:08.083021 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:08.083302 (+ 281us) server_negotiation.cc:407] Connection header received
0807 11:29:08.083494 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:08.083497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:08.083571 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:08.083656 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:08.084630 (+ 974us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45086: BlockingRecv error: recv got EOF from 127.0.0.1:45086 (error 108)
Metrics: {"server-negotiator.queue_time_us":130}
I20260807 11:29:08.128201 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:08.121353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55984 (local address 127.16.197.61:46179)
0807 11:29:08.121653 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:08.121660 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:08.121984 (+ 324us) server_negotiation.cc:407] Connection header received
0807 11:29:08.122243 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:08.122249 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:08.122380 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:08.122488 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:29:08.123026 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:08.124108 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:08.125868 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:08.126590 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:08.126713 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:08.127326 (+ 613us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:29:08.127462 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:29:08.127546 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:08.127709 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 11:29:08.127793 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260807 11:29:08.136811 9122 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:29:08.134550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45092 (local address 127.16.197.60:44381)
0807 11:29:08.134807 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:08.134817 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:08.135220 (+ 403us) server_negotiation.cc:407] Connection header received
0807 11:29:08.135374 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:08.135378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:08.135471 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:08.135555 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:08.136537 (+ 982us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45092: BlockingRecv error: recv got EOF from 127.0.0.1:45092 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 11:29:08.209213 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:08.204174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55998 (local address 127.16.197.61:46179)
0807 11:29:08.204350 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:08.204357 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:08.204733 (+ 376us) server_negotiation.cc:407] Connection header received
0807 11:29:08.204952 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:08.204957 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:08.205029 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:08.205121 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:29:08.205488 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:08.206284 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:08.207691 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:08.208098 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:08.208196 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:08.208597 (+ 401us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:29:08.208713 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:29:08.208780 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:08.208927 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 11:29:08.208993 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260807 11:29:08.217442 9122 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 11:29:08.215081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45094 (local address 127.16.197.60:44381)
0807 11:29:08.215254 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:08.215258 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:08.215725 (+ 467us) server_negotiation.cc:407] Connection header received
0807 11:29:08.215979 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:08.215985 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:08.216100 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:08.216208 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 11:29:08.217087 (+ 879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45094: BlockingRecv error: recv got EOF from 127.0.0.1:45094 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 11:29:08.265153 9099 catalog_manager.cc:1259] Loaded cluster ID: ff3a8ad7f6224300b4491c72fc8016f2
I20260807 11:29:08.265254 9099 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5cd9eb5132804bc9b81cd7d54ea0fd9d: loading cluster ID for follower catalog manager: success
I20260807 11:29:08.267802 9099 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5cd9eb5132804bc9b81cd7d54ea0fd9d: acquiring CA information for follower catalog manager: success
I20260807 11:29:08.269723 9099 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5cd9eb5132804bc9b81cd7d54ea0fd9d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 11:29:08.356240 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:08.351408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56004 (local address 127.16.197.61:46179)
0807 11:29:08.351695 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:08.351702 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:08.351905 (+ 203us) server_negotiation.cc:407] Connection header received
0807 11:29:08.352087 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:08.352091 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:08.352172 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:08.352253 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:29:08.352657 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:08.353341 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:08.354678 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:08.355135 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:08.355239 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:08.355569 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:29:08.355664 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:29:08.355728 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:08.355857 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 11:29:08.355927 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260807 11:29:08.367769 9122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:08.362118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47690 (local address 127.16.197.60:44381)
0807 11:29:08.362346 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:08.362353 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:08.362662 (+ 309us) server_negotiation.cc:407] Connection header received
0807 11:29:08.362894 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:08.362899 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:08.363116 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:08.363243 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:29:08.363714 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:08.364435 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:08.365716 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:08.366108 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:08.366198 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:08.366664 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:29:08.367272 (+ 608us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:29:08.367388 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:08.367484 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 11:29:08.367549 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 11:29:08.384720 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:08.373659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58254 (local address 127.16.197.61:46179)
0807 11:29:08.373819 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:08.373825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:08.378576 (+ 4751us) server_negotiation.cc:407] Connection header received
0807 11:29:08.378801 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:08.378807 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:08.378903 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:08.379030 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:29:08.379734 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:08.380767 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:08.382668 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:08.383258 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:08.383404 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:08.384008 (+ 604us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:29:08.384153 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:29:08.384237 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:08.384417 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 11:29:08.384507 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 11:29:08.384845 9264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:08.373549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52672 (local address 127.16.197.62:33109)
0807 11:29:08.377269 (+ 3720us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:08.377279 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:08.377663 (+ 384us) server_negotiation.cc:407] Connection header received
0807 11:29:08.377924 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:08.377929 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:08.378084 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:08.378269 (+ 185us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:29:08.379231 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:08.380281 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:08.382291 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:08.382789 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:08.382911 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:08.383518 (+ 607us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:29:08.383974 (+ 456us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:29:08.384089 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:08.384244 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 11:29:08.384346 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3541,"thread_start_us":3406,"threads_started":1}
I20260807 11:29:08.386150 9122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 11:29:08.373645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47692 (local address 127.16.197.60:44381)
0807 11:29:08.373818 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 11:29:08.373824 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 11:29:08.379590 (+ 5766us) server_negotiation.cc:407] Connection header received
0807 11:29:08.379797 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 11:29:08.379803 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 11:29:08.379892 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 11:29:08.379979 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 11:29:08.380567 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:08.381671 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:08.383652 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 11:29:08.384302 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 11:29:08.384421 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 11:29:08.385025 (+ 604us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 11:29:08.385169 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 11:29:08.385221 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 11:29:08.385772 (+ 551us) server_negotiation.cc:299] Negotiation successful
0807 11:29:08.385853 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 11:29:08.390283 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 8851
I20260807 11:29:08.391390 8917 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:29:08.504284 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 8851
W20260807 11:29:08.527552 8948 connection.cc:535] server connection from 127.0.0.1:55886 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 11:29:08.527519 9025 connection.cc:535] server connection from 127.0.0.1:44988 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 11:29:08.528196 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 8929
I20260807 11:29:08.529292 8995 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:29:08.638911 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 8929
I20260807 11:29:08.661800 17172 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRFF58w/build/asan/bin/kudu pid 9009
I20260807 11:29:08.663096 9075 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 11:29:08.772116 17172 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRFF58w/build/asan/bin/kudu with pid 9009
2026-08-07T11:29:08Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4641 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9262 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (382799 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST